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Preface 


The  process  of  coordinate  and  compromise  is  a  central,  but 
misunderstood,  feature  of  Air  Force  management.  Although  Air 
Force  civilian  and  military  managers  alike  are  trained  from  the  very 
beginning  of  their  careers  to  become  proficient  in  the  mechanics  of 
the  coordination  process,  the  full  implication  of  what  it  means  to  the 
organization  as  a  whole  is  often  overlooked.  The  inescapable  facts 
are  that  the  Air  Force  is  a  very  large  organization  and  that  it  must 
make  decisions  and  act  upon  those  decisions  to  be  effective.  To 
accomplish  the  functions  of  deciding  and  acting  requires  a  consensus, 
or  agreement,  between  the  key  participants.  To  reach  this 
consensus,  in  turn,  requires  the  communication  of  information. 

In  an  individual,  lucid  thought  and  decisive  action  are  perceived 
of  as  positive  personal  qualities.  In  a  large  organization,  however, 
those  same  qualities  can  be  viewed  instead  as  recklessness  and 

i 

I  naivete'.  Thus,  the  task  of  making  explicit  all  information  affecting 

i' 

an  issue  is  usually  resisted  and  generally  underappreciated.  The 
results  are  that  the  information  used  in  decision  making  is  often  not 
coherent,  that  communication  processes  are  clouded  with  ambiguity 
and  misunderstandings,  and  that  the  ultimate  outcome  of  the 
decision  process  rests  more  on  chance  than  reason. 

The  genesis  of  this  topic  was  my  experience  as  an  acquisition 
program  manager.  While  successful  program  managers  may  already 
practice  some  of  what  is  discussed  in  this  study,  for  others,  like  me, 
the  system  sometimes  seemed  to  present  an  endless  array  of 

li 


hurdles.  This  study  was  my  attempt  to  understand  that.  If  what  I 
have  learned  is  of  some  help  to  another  program  manager  the  study 
will  have  been,  for  me,  a  success. 

For  all  the  positive  aspects  of  this  research,  and  none  of  the 
mistakes,  I  am  indebted  to  several  individuals.  While  at  the  F-16 
System  Program  Office,  Colonel  Bartow  Tucker  and  Lieutenant 
Colonel  Joe  Banks  provided  me  with  an  "on-the-job"  unsurpassed 
education  in  Air  Force  decision  making.  I  thank  them  for  their 
enthusiasm  and  patience.  At  the  Air  Force  Institute  of  Technology,  I 
must  thank  two  of  my  teachers.  First,  Captain  Joe  Tatman,  who 
exposed  me  to  the  principles  and  techniques  of  decision  analysis  and 
struggled  to  help  me  understand  and  apply  them.  Second,  Major 
Larry  W.  Emmelhainz,  who  as  both  a  teacher  and  as  my  thesis 
advisor,  worked  hard  to  help  me  transform  my  many  ideas  into 
focused  academic  research. 

At  home,  I  must  first  acknowledge  the  many  contributions  of 
my  parents,  Bev  and  Vince.  Their  help  in  keeping  the  household 
running  and  in  supporting  my  endeavors  is  greatly  appreciated.  For 
my  wife  Sue,  my  sincerest  love  and  affection  for  providing  me 
unfaltering  encouragement.  Her  critical  ear  and  positive  outlook  kept 
me  solidly  on  course  during  the  last  eight  months.  And  finally,  to 
my  sons  Mark  and  Richard,  thank  you  for  being  there,  for  a  look  at 
your  faces  always  reminded  me  what  it  was  all  about. 
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ABSTRACT 

Program  managers  are  faced  with  the  task  of  integrating  a 
complex  mix  of  goals,  objectives  and  procedures  from  four  sub-pro¬ 
cesses  within  the  major  weapon  system  acquisition  process.  These 
sub-processes  include  needs  determination,  budget,  acquisition  man¬ 
agement,  and  contracts.  This  complexity  is  compounded  by  the  dual 
roles  of  the  program  manager;  traditional  manager  coupled  with 
primary  program  advocate. 

To  be  effective  in  this  organizational  situation,  the  program 
manager  requires  a  framework  to  efficiently  guide  decision  making 
efforts.  This  study  developed  a  normative  decision  making  model  to 
guide  program  managers  in  dealing  with  this  complexity  The 
model,  based  upon  management  theory,  focused  on  information 
generation  and  communication  tasks.  It  was  then  compared  to  the 
Air  Defense  Fighter  Competition  decision. 

Decision  analysis  techniques  were  integrated  into  the  model  to 
structure  the  decision  making  process  to  efficiently  generate  relevant 
information  in  a  form  to  maximize  its  utility  to  the  organization  as 
a  whole. 

The  case  study  evaluated  Air  Defense  Fighter  program 
documentation  to  ascertain  applicability  of  the  model.  Analysis 
showed  that  a  valid  normative  model  can  be  developed  for  use  by  a 


program  manager  working  in  a  bureaucratic  organization,  that  the 
various  sub-processes  generated  some  of  the  information  required  by 
decision  analysis  techniques,  but  that  information  regarding 
organizational  values  and  assessments  of  future  uncertainty  and  risk 
were  not  used  or  requested;  that  coordinating  activities  were  a 
critical  part  of  efforts  to  reach  consensus  concerning  goals,  objectives, 
etc.,  and  that  the  four  sub-processes  did  not,  in  and  of  themselves, 
provide  an  overall  coherent  decision  making  structure. 

Recommendations  were  that  the  Air  Force  undertake  training 
initiatives  in  organizational  decision  making,  that  program  managers 
be  required  to  structure  decision  making  activities  to  generate  and 
document  information  in  accordance  with  the  normative  model  and 
that  methods  to  explicitly  consider  value,  uncertainty  and  risk  be 
studied  and  implemented. 
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THE  DEVELOPMENT  OF  A  NORMATIVE 


ACQUISITION  DECISION  MAKING  MODEL 
INCORPORATING  DECISION  ANALYSIS  PRINCIPLES 


I.  Introduction 


The  Air  Force  acquisition  Program  Manager  must  make  re¬ 
source  allocation  decisions  in  an  organizational  situation  characterized 
by  complex  procedural  directives  while  facing  an  uncertain  and 
changing  environment.  To  accomplish  this  task,  the  Program  Man¬ 
ager  must  effectively  employ  information  search  and  communication 
skills.  While  untried  in  group  process  decision  making,  decision  anal¬ 
ysis  principles  and  techniques  offer  the  opportunity  for  the  Program 
Manager  to  efficiently  focus  acquisition  decision  making  activities. 

This  introductory  chapter  reviews  the  acquisition  decision 
making  environment,  the  current  status  of  policy  as  regards  major 
weapon  system  acquisitions,  and  a  brief  overview  of  existing  man¬ 
agement  tools  used  to  support  management  decision  making.  The 
chapter  goes  on  to  provide  a  problem  statement,  justification  for  the 
research  and  the  research  approach  followed  in  this  study. 


The  Defense  Acquisition  Environment 

Facing  little  or  no  real  budget  growth  for  Fiscal  Years  1988  and 
1989,  and  the  possibility  of  Gramm-Rudman  deficit  reduction  cuts 
(CBO,  1987:  38),  Air  Force  acquisition  managers  in  1986  were  under 
close  cost,  schedule  and  performance  scrutiny.  This  pressure  was 
seen  in  the  increased  interest  that  military  acquisition  was  receiving 
as  a  result  of  media  attention.  During  the  last  several  years  the 
news  media  has  reported  on  the  "spare  parts  folly"  (Bernstein,  1984: 
123)  and  the  inherent  management  inefficiencies  of  the  "military- 
Congressional  complex"  (Fossedal,  1985:  22).  In  each  case  attention 
was  directed  at  how  the  government  managed  its  military  acquisi¬ 
tions. 

The  government  agencies  responsible  for  the  management  of 
the  military  responded  to  these  concerns,  fn  testimony  before  the 
Congress  in  February  1986,  Secretary  of  Defense  Casper  W.  Weinberg¬ 
er  stated  that  a  major  priority  of  the  new  Defense  budget  was  good 
management  (Weinberger,  1986:  19).  Good  management  in  this  con¬ 
text  focused  on  the  process  of  Defense  management,  not  just  its  out¬ 
comes.  Given  such  demands,  the  quality  of  the  acquisition  decision 
making  process  became  increasingly  critical. 

In  addition,  President  Reagan's  Blue  Ribbon  Commission  on  De¬ 
fense  Management  (the  Packard  Commission)  concluded  that  signifi¬ 
cant  changes  were  required  to  improve  the  overall  effectiveness  of 
management  processes  in  the  Department  of  Defense  (DOD).  One  of 
its  four  main  recommendation  areas  concerned  the  acquisition  of 
weapon  systems.  The  Commission's  recommendation  dealt  with  the 


manner  in  which  the  acquisition  process  could  be  made  more  effec¬ 
tive  and  efficient,  in  a  business  sense,  and  thus  result  in  a  better 
product  at  less  cost  to  the  taxpayers  (President’s ....  1986:  xxiv- 
xxvii). 

Acquisition  Policy 

Effective  and  efficient  management  of  the  weapon  system  ac¬ 
quisition  process  was  not,  however,  first  identified  by  the  Packard 
Commission.  A  1969  pamphlet  published  by  the  DOD  Defense  Systems 
Management  College  (DSMC),  explored  the  nature  of  the  acquisition 
program  manager's  (PM)  management  role  in  depth.  The  PM’s  stat¬ 
ed  role  was 

...  to  tie  together,  to  manage,  to  direct  the  development 
and  production  of  a  system  meeting  performance,  sched¬ 
ule,  and  cost  objectives  which  are  defined  by  his  Service 
and  approved  by  the  Secretary  of  Defense  (SECDEF) 

(DSMC,  1969:  2). 

To  accomplish  this  task,  the  PM  must  effectively  create  a 
'program  balance'  to  efficiently  reconcile  all  operational  requirements 
and  resource  constraints  having  an  impact  upon  the  program  (DSMC, 
1969:  7). 

DOD  policy  directives  on  the  acquisition  of  major  weapon  sys¬ 
tems  echoed  this  tasking  for  the  program  manager.  In  Department 
of  Defense  Directive  5000.1,  Major  System  Acquisitions,  the  program 

manager  was  given  the  principle  and  objective  of  achieving  a  "cost- 
effective  balance"  between  the  life  cycle  costs  of  the  system  and  its 
mission  effectiveness  (DOD,  1986:  2).  Numerous  other  examples  of 
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regulatory  guidance  were  in  place  which  defined  the  ways  that  a 
PM  must  deal  with  programmatic  and  technical  issues.  These  guide¬ 
lines  came  from  all  corners  of  the  government  including  the 
Congress,  the  Office  of  Management  and  Budget  (0MB),  the  Offices  of 
the  Secretaries  of  Defense  and  Air  Force,  the  Headquarters  Air  Force 
staff,  and  the  major  commands. 

The  0MB,  for  example,  has  directed  that  acquisition  program 
managers  for  each  major  system  must  " . . .  ensure  appropriate 
trade-off  among  investment  costs,  ownership  costs,  schedules,  and 
performance  characteristics"  (0MB,  1976:  4).  Air  Force  Regulation 
(AFR)  800-2,  Acquisition  Program  Management,  assigns  to  the  PM 

the  responsibility  to  "...  make  management  decisions  .  equally 
weighing  cost,  schedule,  performance  and  supportability"  (DAF,  1985: 
6).  Given  the  magnitude  and  complexity  of  the  organizations  with 
which  the  PM  must  deal,  the  success  of  those  decisions  is  influenced 
by  his  or  her  ability  to  communicate  effectively  (Drucker,  1967:  66- 
67). 

The  Program  Manager  had  come  to  assume  a  dual  role.  The 
PM  was  both  a  'manager'  and  an  advocate'  for  the  program  (Fox, 
1974:  460).  As  the  'manager'  of  the  program,  the  PM  was  responsi¬ 
ble  for  matching  program  output  to  the  larger  organizational  needs 
of  the  Air  Force  and  managing  the  actual  conduct  of  the  program. 
The  PM  had  to  "...  oversee  the  Department's  effort  to  acquire,  de¬ 
ploy,  operate,  and  support  major  weapon  systems  of  proven  capabili¬ 
ty,  within  approved  schedules  and  budgets”  (Fox,  1974:  173).  At  the 
same  time,  as  the  program  advocate',  the  Program  Manager  was 
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responsible  for  "...  promoting]  and  defending]  his  program  before 
other  levels  of  the  Defense  Department  and  before  Congress"  (Fox, 
1974:  178).  To  defend  the  program,  the  PM  had  to  project 
"unwavering  optimism"  to  avoid  program  budget  reductions  that 
might  jeopardize  the  program  (Fox,  1974:  178,  180).  This  motivation 
of  the  Program  Manager  was  compounded  by  perceptions  that  the 
promotion  potential  of  the  PM  was  enhanced  by  budgetary  successes 
which  avoided  orogrammatic  reductions  (Kanter,  1979:  114  and  Fox, 
1974:  180). 

In  both  of  these  roles,  the  Program  Manager  was  forced  to  rely 
upon  the  efforts  of  others  The  PM  was  responsible  for  defining  the 
program  objectives  and  ensuring  they  were  achieved.  This  implied 
"controlling  and  coordinating  the  work  [of  others]  so  that  no  one  as¬ 
pect  dominate[d]  others  to  the  detriment  of  the  harmony  of  the 
whole"  (DSMC,  1969:  2)  Therefore,  part  of  this  dual  role  as  both  the 
program  s  manager  and  its  chief  advocate,  the  most  important  func¬ 
tion  of  the  PM  was  said  to  be 

getting  people  to  communicate  with  each  other  to 
achieve  a  common  understanding  of  the  needs  of  the  pro¬ 
gram  and  their  place  in  the  harmony  of  the  total  pro¬ 
gram  effort  (DSMC,  1969:  2). 

The  practical  objective  of  this  communication  effort  was  to  en¬ 
sure  that  the  program  proceeded  as  smoothly  as  possible  by  balanc¬ 
ing  the  needs  of  the  program  with  the  needs  of  the  larger  organiza¬ 
tion.  According  to  Wiener,  "...  communication  is  the  cement  that 
makes  organizations  [Emphasis  in  original].  Communication  alone 
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enables  a  group  to  think  together,  to  see  together,  and  to  act 
together”  (Wiener,  1955:  40). 

What  is  communicated  by  the  PM  is  information.  This  infor¬ 
mation  must  first  have  been  transformed  from  simple  data  to  be 
useful  to  recipients.  Information  is  created  by  selecting  and  struc¬ 
turing  data  with  respect  to  a  specific  problem,  user,  time,  and  place 
(Ebert  &  Mitchell,  1975:  96,  Schoderbek,  et  al,  1985:  153).  A  common 
formula  states  that  when  A  communicates  something  with  B  then  X 
is  the  result:  A  -»  B  ■  X  (Schoderbek,  et  al,  1985:  167-168).  However, 
Thayer  states  that  the  information  communicated  is  a  product  of  the 
receiver  as  well  as  the  sender  (Thayer,  1968:  48-51) 

As  the  Air  Force  has  become  more  functionally  specialized  due 
to  the  increasing  complexity  of  war  fighting  technology,  staff  man¬ 
agers  have  become  more  abundant  and  powerful  (Harrison,  1978: 
577-578).  For  the  PM,  as  primarily  a  line  manager  (i.e.  responsible 
for  accomplishing  core  organizational  tasks),  the  informational  needs 
of  the  staff  must  then  be  taken  into  account  to  minimize  mtra-orga- 
mzational  conflict  (Harrison,  1978:  587). 

Unfortunately  for  the  PM,  the  definition  of  what  constitutes 
useful  information  can  often  vary  depending  on  one's  organizational 
perspective.  For  example,  the  staff  organization  at  a  headquarters 
will  require  'information'  that  directly  supports  their  specific  respon¬ 
sibilities  while  the  PM  will  want  to  provide  'information'  that  sup¬ 
ports  the  immediate  needs  of  the  program  (Fox,  1974:  175-176). 

When  these  points  of  view  diverge,  the  communication  skills  of  the 
PM  are  tested. 


This  situation  is  made  more  complex  by  the  very  nature  of  the 
system  within  which  the  Program  Manager  must  work.  Instead  of 
a  single  process  to  manage,  the  PM  is  actually  confronted  with  sev¬ 
eral  contributing  'sub-processes'  that  feed  the  overall  weapon  system 
acquisition  system  (DAF,  1976:  1).  Four  critical  sub-processes  are 
determining  requirements,  budgeting,  contracting  and  the  overall 
managing  of  the  acquisition  program.  The  PM  must  ensure  that  the 
many  functions  required  by  these  sub-processes  are  identified,  tai¬ 
lored  and  augmented  as  needed,  and  "integrated  into  an  overall 
management  organization  and  approach  that  will  ensure  a  successful 
acquisition  effort  to  achieve  program  objectives”  (DAF,  1976:  4). 

What  is  required  is  to  embue  the  information  to  be  communi¬ 
cated  with  an  unabiguous  and  precise  meaning  which  is  shared  by 
both  the  sender  and  receiver  while  satisfying  the  procedural  re¬ 
quirements  of  the  various  acquisition  sub-processes.  Ideally,  to  best 
serve  the  needs  of  the  overall  organization,  an  unbiased,  structured, 
and  objective  decision  making  process  is  needed  that  provides  the 
best  available  information  (Harrison,  1978:  586-588  and  Fox,  1974: 

180).  The  Air  Force  currently  lacks  an  overall  framework  to  help  the 
PM  efficiently  balance  the  various  sub-process  goals,  objectives  and 
rules. 


Existing  Management  Tools 

Many  decision  making  systems  have  been  provided  to  the  Air 
Force  PM  over  the  years.  These  tools  have  been  designed  and 


offered  as  the  means  to  accomplish  the  data-to-mformation  transfor¬ 
mation  process  and  to  support  subsequent  communications.  These 
tools  have  often  been  designed  to  provide  specific  answers  to 
problems  that  can  be  easily  quantified  and  subjected  to  computer 
analysis.  Post  World  War  II  operations  research  tools  (e  g. 
optimization  techniques  such  as  linear  programming)  were  effective 
primarily  with  operationally  repetitive  problems.  In  the  1950's, 
management  science  developed  to  address  high  level  management 
problems.  These  efforts,  however,  tended  to  focus  upon  problems 
that  could  be  solved  with  elegant  mathematics  (Matheson  and 
Howard,  1968:  21-22). 

Optimization  techniques  and  elegant  mathematics  required  that 
simplifying  assumptions  be  made  concerning  both  the  nature  of  the 
decision  to  be  made  and  the  operational  characteristics  of  the  system 
under  examination.  Three  of  the  most  significant  assumptions  were 
that  an  objective  function  could  be  formed  containing  a  single  at¬ 
tribute  for  evaluation,  that  outcomes  were  not  subject  to  uncertain¬ 
ty  (de  Neuville  and  Stafford,  1971:  87)  and  that  decision  makers  were 
utility  maximizers  (Simon,  1978:  2-6  and  Simon,  1979:  498-499). 

The  same  was  true  of  computer  simulations.  These  highly 
complex  computer  programs  included  many  implicit  and  often  subtle 
assumptions  that  affected  data  provided  by  the  simulations 
(Shannon,  1975:  298-299).  As  program  information  was  presented  to 
groups  further  and  further  removed  from  the  raw  simulation  data 
upon  which  they  were  originally  based,  the  clarity  of  the  intended 


message,  in  terms  of  assumptions,  limitations,  and  risks,  was  dimin¬ 
ished  (Simon,  1959:  265-273). 

The  assumptions  employed  in  the  tools  of  management  science 
are  necessary  because  the  real  world  is  complex  and  dynamic. 

Herbert  Simon  stated  that  even  with  these  powerful  new  tools, 

"most  real-life  choices  still  lie  beyond  the  reach  of  maximizing 
techniques  -  unless  the  situations  are  heroically  simplified  by  drastic 
approximations  "  (Simon,  1959:  259).  The  real  world  and  its 
problems  are  not  simple  and  straight  forward.  They  are  messy. 
According  to  Howard  Raiffa,  there  is  an  art  rather  than  a  science  to 
analyzing  real  world  problems.  The  'art1  required  is  the 
understanding  of  the  decision  maker's  perceptions  of  the 
environment  and  how  he  or  she  manages  the  process  of  choice 
(Raiffa,  1968:  239). 

What  is  needed  then  is  a  normative  framework  that  allows 
the  PM  to  bring  together  data  and  to  efficiently  transform  the  data 
into  information  in  such  a  way  that  the  PM  can  then  effectively 
communicate  the  information  to  others.  The  tools  of  management 
science  have  not  accomplished  that  task  and  organizational  structure 
and  formal  procedures  by  themselves  are  insufficient.  The  principles 
and  techniques  of  decision  analysis,  however,  offer  the  opportunity  to 
deal  with  these  problems. 

Decision  analysis  includes  procedures  and  methodology  for 
assessing  the  real  nature  of  a  situation  in  which  a  deci¬ 
sion  might  be  made,  for  capturing  the  essence  of  that 
situation  in  a  formal  but  transparent  manner,  for  for¬ 
mally  ‘solving'  the  decision  problem,  and  for  providing  in- 


sight  and  motivation  to  the  decision-makers  and  imple¬ 
mented  (Howard  and  Matheson,  1983:  viii). 

For  example,  decision  analysis  principles  and  techniques  were 
used  to  help  evaluate  alternatives  in  the  early  stages  of  the  Mars 
exploration  program.  Decision  analysis  was  beneficial  in  that  it  de¬ 
fined  a  simplified  version  of  the  decisions  to  be  made,  which  then 
could  be  improved  incrementally,  into  the  more  complex  and  realistic 
models  needed  to  decide  upon  an  actual  mission.  Different  alterna¬ 
tives  were  defined  and  then  two  fundamental  operations  were  per¬ 
formed:  expectation  and  maximization.  These  helped  determine,  in 
the  face  of  extreme  complexity  and  uncertainty,  the  most  economical 
decision  (Matheson  and  Howard,  1968:  48). 

Similarly,  for  acquisition  Program  Managers  faced  with  in¬ 
creased  scrutiny  of  their  programs,  decision  analysis  techniques  could 
improve  the  quality  of  decision  related  information  and  thus  increase 
the  effectiveness  of  the  communication  process.  Decision  analysis 
principles  directly  address  the  two  key  issues  in  organizational  deci¬ 
sion  making:  efficient  information  generation  and  effective  informa¬ 
tion  communication.  Ultimately,  use  of  decision  analysis  techniques 
could  potentially  increase  the  defensibility  of  programmatic  decisions 
as  being  rational,  informed  and  consistent  (Raiffa,  1968:  266-269). 

Statement  of  the  Problem 

Current  rules  and  procedures  for  decision  making  by  acquisition 
Program  Managers  in  the  Air  Force  constitute  a  complex  approach 
for  allocating  the  resources  under  their  control  This  complexity  has 


resulted  in  a  fragmentation  of  objectives  and  rules  with  which  the 
PM  must  deal.  This  fragmentation  affects  the  ability  of  the  Air 
Force  to  efficiently  choose  a  course  of  action  and  then  to  effectively 
defend  or  justify  that  choice  to  others.  Efficiency  requires  a  struc¬ 
tured  information  generation  process  while  effectiveness  requires  an 
explicit  and  unambiguous  communication  of  relevant  facts  The  Air 
Force  currently  lacks  an  overall  conceptual  framework  by  which  or¬ 
ganizationally  fragmented  decision  making  rules  and  procedures  can 
efficiently  and  effectively  guide  the  PM.  While  decision  analysis  ap¬ 
pears  to  offer  a  means  of  solving  this  problem,  the  technique  has  not 
been  applied  to  situations  analogous  with  that  of  the  weapon  system 
acquisition  Program  Manager. 

Objective  o[  the  Study 

The  objective  of  this  research  is  to  provide  better  understand¬ 
ing  of  decision  making  processes  in  the  Air  Force  acquisition  arena  in 
terms  of  the  information  generated  and  how  that  information  is 
subsequently  communicated  to  others  and  ultimately  acted  upon. 
Specifically,  the  intent  of  this  study  is  to: 

1.  develop  a  normative  decision  making  mod¬ 
el  for  use  in  the  weapon  system  acquisition 
management  process  which  integrates  decision 
analysis  principles  and  procedures, 


2.  test  this  model  against  an  actual  program 
management  decision,  and 


3.  draw  conclusions  about  the  implementa¬ 
tion  of  the  model  into  acquisition  manage¬ 
ment  procedures. 

Justification 

The  use  of  decision  analysis  techniques  in  conjunction  with  an 
overall  normative  decision  making  model  can  assist  Air  Force  Pro¬ 
gram  Managers  in  allocating  the  scarce  resources  of  money,  time, 
and  manpower  effectively  and  efficiently.  Decision  analysis  inher¬ 
ently  provides  structure  to  data.  This  structure  increases  under¬ 
standing  of  the  problem  and  clarifies  what  decision  is  to  be  made, 
thus  focusing  the  PM's  decision  making  process.  This  focus  should 
improve  the  ability  of  the  PM  to  transmit  decision  information  using 
available  communication  channels.  Decision  analysis  principles  help 
ensure  that  decisions  are  adequately  defined,  rationally  consistent 
and  defensible  to  others.  To  the  extent  that  these  principles  can  be 
used  with  the  complex  decisions  and  inter-related  management  pro¬ 
cesses  faced  by  the  PM,  the  overall  management  of  the  weapon  sys¬ 
tem  acquisition  process  should  be  improved. 

Approach 

The  primary  work  necessary  to  accomplish  the  study  objective 
is  divided  into  two  parts.  First,  in  Chapter  IV  the  results  of  the  lit¬ 
erature  review  are  used  to  create  a  model  of  organizational  decision 
making  which  incorporates  the  principles  of  decision  analysis.  This 
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model  is  intended  as  a  general  decision  making  framework  and  is 
called  the  Organizational  Decision  Analysis  Model  (ODAM). 

Second,  in  Chapter  V  a  review  of  a  recent  Air  Force  acquisition 
decision  (the  Air  Defense  Fighter  Competition)  is  performed  and  the 
results  compared  to  the  basic  attributes  of  the  ODAM.  The  purpose 
of  this  comparison  is  to  identify  the  extent  to  which  the  generalized 
model  might  be  appropriate  for,  and  improve  decision  making  in,  the 
specific  organizational  setting  of  the  DOD  weapon  system  acquisition 
process.  The  detailed  methodology  of  this  comparison  is  covered  in 
Chapter  III.  Figure  1  summarizes  the  study  approach  and  the 
products  produced. 

Summary 

This  chapter  defined  the  problem  to  be  studied  as  one  con¬ 
cerned  with  the  information  used  in  the  weapon  system  acquisition 
management  process  by  the  Program  Manager.  The  use  of  this  in¬ 
formation  in  support  of  organizational  decisions,  and  its  communica¬ 
tion  to  that  end,  are  central  to  the  research.  The  method  used  in 
the  study  encompasses  stages  described  in  terms  of  a  two  part  ap¬ 
proach.  Chapter  IV  produces  a  generalized  model  (ODAM)  based  upon 
the  results  of  the  literature  review.  Chapter  V  then  compares  the 
ADF  Competition  to  the  major  attributes  of  the  ODAM.  Finally,  in 
Chapter  VI  the  study  is  summarized  and  conclusions  are  drawn. 
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II.  Literature  Review 


Introduction 

The  purpose  of  this  chapter  is  to  review  the  literature 
associated  with  the  topic  of  organizational  decision  making  in  general 
and  Air  Force  acquisition  decision  making  in  particular.  The  next 
section  provides  a  broad  background  of  decision  making  as  a  topic 
relevant  to  the  study  of  management.  In  the  third  section,  the  DOD 
acquisition  decision  making  structure  and  procedures  are  defined  and 
the  current  acquisition  environment  is  reviewed.  This  organizational 
structure  is  shown  to  be  comprised  of  four  independent,  but  inter¬ 
related,  sub-processes.  In  the  fourth  section,  management  theory  is 
reviewed  as  it  relates  to  organizational  decision  making.  The 
management  theory  section  covers  the  overall  decision  making  task 
in  terms  of  organizational  structure  and  organization  effectiveness. 
There  is  also  a  discussion  of  three  types  of  management  organization 
and  how  these  impact  decision  making.  Finally,  decision  analysis  is 
discussed,  its  approach  defined  in  detail,  and  its  major  techniques 
briefly  explained. 

Background 

The  basic  economic  problem  is  to  allocate,  at  a  given  point  in 
time,  relatively  scarce  resources  among  relatively  infinite,  and 
competing,  culturally  determined  needs  (Liebhafsky,  1968:  13).  The 
managerial  corollary  is  that  rational  resource  allocation  increases  the 
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overall  effectiveness  of  organizations  by  efficiently  satisfying  needs: 
that  is,  rational  decision  making  provides  for  the  productive 
utilization  of  resources  (Drucker,  1974:  67).  Insofar  as  managers 
affect  action  through  organizations,  the  following  observation  by 
Baligh  (1986:  1488)  is  applicable: 

Organizations  are  created  because  people  have  bounded 
capacities  for  making  decisions,  and  organizations  allow 
specialization,  decomposition,  etc.,  which  allow  one  to  fit 
the  work  to  the  decision  making  capabilities  of  the  people. 

This  implies  that  organizations  stretch  the  decision  making 

capacities  of  their  people.  Organizations  stretch  the  capacities  of  their 

people  through  institutionalized  decision  making  processes.  These 

processes  are  logical  relations  of  people  that  encourage  the 

communication  of  relevant  information  in  support  of  the  decision 

(Baligh,  1986:  1481). 

Communication  has  been  likened  to  the  cement  that  holds  an 
organization  together  (Thayer,  1968:  48-51).  This  is  accomplished 
through  the  transmission  of  information.  In  this  role  of  support  to 
the  decision  making  process,  information  must  be  treated  as  a 
resource  along  with  land,  labor  and  capital.  Information,  then,  as  " 

...  a  resource  ...  must  be  treated  as  such  -  a  resource  having 
costs  and  benefits  associated  with  it"  (Schoderbek,  et  al,  1985;  154). 

The  use  of  information  during  a  decision  making  process  must  then 
be  guided  by  marginal  analysis.  Marginal  analysis  concludes  that 
additional  information  will  be  acquired  until  the  costs  of  the 
additional  information  exceed  its  benefits  (Schoderbek,  et  al,  1985: 
154-159). 
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During  the  Air  Force  process  for  enacting  organizational 
resource  allocation  decisions,  primarily  in  the  major  weapon  system 
acquisition  process,  information  is  not  specifically  addressed  as  a 
resource.  In  decision  theory,  however,  information  is  a  valuable 
commodity  as  are  the  other  traditional  resources  (personnel, 
equipment,  raw  materials,  etc.).  Therefore,  when  discussing 
acquisition  decision  making,  information  will  be  viewed  as  a  scarce 
resource  which  should  be  used  prudently  As  the  discussion  develops 
in  this  chapter,  the  focus  will  be  on  the  organization  and  the 
acquisition  Program  Manager,  organizational  processes  for  generation 
and  communication  of  decision  relevant  information,  and  the 
Program  Manager's  role  throughout  that  process. 

The  POD  Acquisition  Process 

In  most  discussions  of  the  problems  confronting  the  Air  Force 
major  weapon  system  acquisition  process,  there  has  been  an 
inescapable  tendency  to  focus  on  individual  results.  This  was 
especially  true  of  the  media  and  to  a  lesser  degree,  the  Congress. 
While  understandable  why  the  issue  of  a  $9,600  alien  wrench 
triggered  a  review  of  acquisition  procedures,  it  is  not  a  suitable 
example  upon  which  to  base  findings  and  recommendations  affecting 
the  purchase  of  a  new  tactical  fighter  or  the  development  of  an 
advanced  air-to-air  radar.  Secretary  of  Defense  Casper  Weinberger 
recently  said  that  it  is  "  ...  alarming  to  see  an  issue  as  important 
as  the  proper  level  of  American  defense  preparedness  hanging  on 
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questions  such  as  whether  departmental  practices  have  resulted  in 
overpayment  for  a  hammer  or  an  airplane  toilet  seat  as  if  a  system 
of  zero  errors  were  the  goal  ...  “  (Weinberger,  1986;  21). 

More  to  the  point,  Hiatt  and  Atkinson  believed  that  if  everyone 
associated  with  the  development  and  acquisition  of  major  weapon 
systems  was  "...  lost  in  the  trees  of  program  management,  unable 
to  see  -  far  less  influence  -  the  policy  forest  ...  "  (Hiatt  &  Atkinson, 
1985.  7),  meaningful  changes  to  overall  conduct  of  the  weapon 
system  acquisition  process  would  likely  not  occur.  To  successfully 
alter  the  direction  of  weapon  system  acquisitions  required  a 
systematic  and  comprehensive  review  of  all  elements  associated  with 
the  process  -  particularly  how  decisions  are  made.  A  focus  on  micro- 
decisions  (i.e.  decisions  made  by  an  individual  manager)  within  a 
single  management  process  increases  the  risk  associated  with  those 
decisions  and  to  the  meeting  of  the  objectives  of  the  weapon  system 
acquisition  process  as  a  whole  (Hiatt  &  Atkinson,  1985:  7). 

These  environmental  forces  influenced  the  nature  of  the  overall 
acquisition  task.  The  environment  is  defined  as  those  factors  outside 
the  control  of  the  organization  but  which  materially  affect  how  the 
organizations  perform  (Churchman,  1968  36  and  Schoderbek,  et  al 
1985,  199-200).  For  this  study,  organizations  outside  the  Department 
of  the  Air  Force  that  materially  affect  the  management  of  the 
development  and  acquisition  of  major  weapon  systems  constitute  the 
environment.  To  aid  in  this  discussion,  only  two  are  addressed  here 
other  Executive  Branch  agencies  outside  the  Department  of  the  Air 
Force  and  the  United  States  Congress.  After  a  review  of  the 


structure  and  procedures  of  acquisition  management,  examples  of 
recent  environmental  changes  will  be  discussed  in  terms  of  their 
impact  upon  decisions  made  by  the  Program  Manager  during  the 
weapon  system  acquisition  processes. 

The  Structure  of  Acquisition  Management.  There  is  a  long¬ 
standing  tradition  when  discussing  the  weapon  system  acquisition 
system  to  immediately  dissect  the  whole  into  easily  managed  parts 
(For  example  see  Fox,  1974,  Gansler,  1980  and  DAF,  1976)  More 
recently,  Mullins  stressed  the  importance  of  the  interdependent 
nature  of  defense  acquisition  activities  and  their  participants  (1986 
113).  While  following  the  traditional  approach  on  the  surface,  the  key 
facet  to  be  brought  out  of  this  study  is  the  interdependence  of  the 
individual  parts  of  the  overall  DOD  acquisition  system.  Therefore, 
the  decisions  that  occur  during  the  product  life-cycle  of  a  major 
weapon  system  have  been  organized  into  four  sub-processes  (i.e 
parts'):  the  requirements  process,  the  budget  process,  the  major 
weapon  system  acquisition  process,  and  the  contracting  process  As 
pointed  out  by  Mullins,  each  of  these  sub-processes  is  distinct  but,  at 
the  same  time,  interdependent  -  a  change  to  one  often  affects  the 
others,  sometimes  in  subtle,  yet  significant,  ways. 

The  Operational  Requirements  Sub-Process.  Through  the 

operational  requirements  sub-process 

operational  needs  are  formulated  and  related 
research,  development,  and  acquisition  (RD&A)  programs 
are  defined  and  undertaken  From  these  and  succeeding 
programs,  new  and  improved  weapon  systems  emerge 
(DAF,  1985:  2). 
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Implicitly  required  during  this  sub-process  is  coordination 
throughout  the  Air  Force  that  takes  place  prior  to  a  requirement 
being  ’validated’.  A  validated  Statement  of  Operational  Need  (SON), 
the  documented  requirement,  is  an  agreed  to  position  by  the 
corporate  Air  Force  that  an  additional  capability  is  required  to 
facilitate  the  accomplishment  of  a  stated  Air  Force  mission.  As 
stated  in  Air  Force  Regulation  57-1,  Operational  Needs: 

The  SON  has  three  principle  uses:  It  defines  an 
operational  need,  obtains  official  validation  of  the  need, 
and  furnishes  preliminary  guidance  for  RD&A  planning  by 
the  responsible  implementing  and  participating  commands 
(DAF,  1985:  2). 

Ideally,  the  military  establishes  missions  based  upon  guidance 
provided  from  the  civilian  defense  leadership  which  is  headed  by  the 
President.  This  guidance  takes  the  form  of  stated  national  objectives 
which  require,  to  varying  degrees,  military  support.  Given  these 
overall  national  objectives,  and  the  accompanying  military  strategy 
deemed  appropriate,  the  military  then  formulates  a  program  to 
provide  the  armed  forces  necessary  to  accomplish  the  required  tasks 
(Meehan,  1985). 

The  preparation  of  a  SON  is  the  first  step  in  the  long  process  of 
transforming  a  military  requirement  necessary  to  fulfill  a  national 
objective  into  a  reality  employable  by  the  armed  forces  -  in  this  case 
the  Air  Force.  SON'S  may  be  initiated  by  any  major  command  or 
higher  headquarters  in  the  Air  Force.  Usually  they  represent  either 
an  identified  operational  deficiency  or  a  technological  opportunity 


(McCarty,  1986:  5).  Opportunities  arise  from  technological  advances. 
Operational  deficiencies  come  from  one  of  three  sources  (DAF,  1985  3). 

1.  Deficiencies  caused  by  changes  in  the  threat. 

2.  Deficiencies  resulting  from  a  redefinition  of  the 
assigned  task. 

3.  Deficiencies  stemming  from  a  deterioration  in 
operational  performance  (e  g.  aging  systems). 

Perhaps  the  SON'S  most  important  function,  however,  is  that  it 
represents  the  first  time  in  the  acquisition  life-cycle  that  a  corporate 
Air  Force  decision  is  made  concerning  a  possible  new  program.  The 
coordination  process  required  of  a  SON  is  lengthy  and  involves  most 
segments  within  the  Air  Force.  After  a  need  is  identified  and 
documented  in  a  prescribed  format  for  the  SON,  the  SON  is  submitted 
first  to  the  other  Major  Commands  for  review  and  comment  and 
then  to  the  Air  Staff.  Air  Force  Systems  Command  (AFSC)  and  Air 
Force  Logistics  Command  (AFLC)  both  comment  from  the  perspective 
of  developer  and  main  tamer.  Other  operational  commands  comment 
on  joint  requirements  or  possible  redundancies. 

The  Air  Staff  responds  to  the  submission  by  selecting  an  Office 
of  Primary  Responsibility  (OPR)  and  initiating  the  Air  Staff 
coordination  cycle.  First,  the  SON  is  assessed  to  determine  "...  if  the 
operational  need  is  clear  and  satisfactorily  substantiated  and  if  the 
proposed  program  is  sound  and  a  suitable  response  to  the  need"  (DAF, 
1985  13).  This  function  is  performed  primarily  by  a  General  Officer 
panel  called  the  Requirements  Review  Group  (RRG).  Validation  of 
the  SON  by  the  RRG  implies  that  ”...  the  proposed  program  is 


sound  and  can  compete  for  funding  in  the  POM  process  (DAF,  1985: 
13).  As  the  operational  requirements  sub-process  proceeds  into  the 
budget  sub-process,  a  Justification  for  Major  System  New  Start 
(JMSNS)  is  required  for  a  major  system  (A  major  system  is  one 
defined  in  accordance  with  criteria  set  forth  in  DODI  5000.2  -  Major 
Systems  Acquisition).  The  requirements  sub-process  is  summarized 
in  Figure  2. 

The  SON  validation  process  is  the  Air  Force  method  to  ensure 
that  new  programs  are  truly  needed  and  are  the  right  programs 
before  attempting  to  tackle  the  even  more  lengthy  task  of  securing 
the  needed  resources  to  conduct  the  program.  The  requirements 
process  is  an  organized  set  of  micro-decisions  aimed  at  determining 
whether  a  new  program  is  appropriate  to  initiate.  The  primary 
communication  channel  is  the  coordination  between  major 
commands  headquarters  and  the  Air  Staff  coordination  control  by 
the  RRG.  There  is  no  role  for  the  Program  Manager  in  the 
requirements  process  since  the  program  has  not  yet  begun,  no 
manager  has  yet  been  appointed.  AFSC's  contribution  is  limited  to 
providing  information  on  technological  considerations  that  may 
contribute  to  satisfying  the  need  (DAF,  1985:  16). 

The  Budget  Sub-Process.  The  'budget  process'  not  only 

involves  accounting  for  funds  but  also  is  the  systematic  way  in 
which  the  corporate  Air  Force  makes  resource  allocation  decisions. 
The  system  to  accomplish  the  budget  process  is  the  Planning, 
Programming  and  Budgeting  System  (PPBS).  The  PPBS  is  "...  the 
DOD  resource  management  system  . . .  [and]  its  purpose  is  to  identify 
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needs,  match  them  with  resource  requirements,  and  translate  them 
into  budget  proposals"  (DAF,  1987  2). 

In  general,  "...  planning  refers  to  decisions  by  the  organization 
about  what  results  are  desired  and  how  the  results  should  be 
accomplished  (Anthony  and  Young,  1984:  4).  Strategic  planning  deals 
with  "...the  futurity  of  current  decisions"  (Steiner,  et  al,  1982:  172) 
Corporate  planning  in  the  business  world: 

. .  may  be  defined  as  a  structure  of  plans.  It  is  a 
structure  that  integrates  strategic  with  short  range 
operational  plans.  In  this  structure  are  integrated,  at  all 
levels,  major  objectives,  strategies,  policies,  and  functions 
of  an  enterprise  (Steiner,  et  al,  1982:  173). 

Programming  decisions  are  those  made  "...  with  respect  to  the 

major  programs  the  organization  plans  to  undertake  during  the 

coming  period"  (Anthony  &  Young,  1984:  11).  Programming  is  the 

bridge  between  the  plans  and  their  implementation. 

Finally,  "...  a  budget  is  a  plan  expressed  in  monetary  terms" 

that  identifies  the  organization  element  responsible  for  the  task 

accomplishment  (Anthony  &  Young,  1984:  357).  Budgeting  is  an 

essential  part  of  management  control. 

The  PPBS  is  a  dynamic  and  evolving  system  whose  focus  and 

procedures  are  sensitive  to  changes  in  key  personnel  and  policy 

direction.  "One  of  the  greatest  single  sources  of  change  is  the  seating 

of  a  new  political  administration.  Each  new  Secretary  of  Defense 

adjusts  the  system  to  reflect  his  own  style  of  management"  (DAF, 

1987:  2). 


In  the  current  administration,  for  example,  one  of  the  areas  of 
emphasis  relevant  to  the  decision  making  process  is  'participatory 
management'  This  concept  deais  with  the  decision  making  process 
by  defining  who  the  players  are.  Recently,  the  role  of  the 
operational  Commanders  in  Chief  (CinC)  and  the  Joint  Chiefs  of  Staff 
(JCS)  has  been  strengthened  (Carlucci,  1981:  3). 

As  the  name  implies,  there  are  three  phases  in  the  PPBS. 
planning,  programming  and  budgeting  (DAF,  1987:  8)  The  DOD 
planning  phase  complements  the  requirements  validation  process 
outlined  above.  It  is  designed  to  document  the  forces  required  to 
counter  the  currently  defined  threat  within  the  constraints  of  the 
Air  Force  mission  statement.  The  planning  phase  is  the  benchmark 
that  highlights  critical  needs  and  examines  risks.  It  differs  from  the 
requirements  process  in  that  it  deals  with  all  needs,  not  just  newly 
defined  requirements. 

The  key  document  produced  during  the  planning  phase  of  the 
PPBS  is  the  Defense  Guidance  (DG).  The  DG 

.  .  is  designed  to  guide  resource  allocation  decisions  which 
occur  during  the  programming  and  budgeting  phases 
Services  develop  their  program  proposals  in  accordance 
with  it  while  OSD  and  Joint  Staff  use  it  as  the  baseline 
for  program  review  (DAF,  1987:  14) 

As  the  Secretary  of  Defense  guidance  to  all  DOD,  the  DG 

addresses  policy  guidance,  strategy,  force  planning,  resource  planning, 

fiscal  guidance  and  major  issues.  Although  the  DG  is  primarily  the 

SECDEF’s  direction  to  the  DOD,  all  elements  within  the  Defense 

Department  have  the  opportunity  to  input  to  it.  "The  Air  Force  has 


two  channels  for  input  [to  the  DG]  -  first  directly  to  OSD  and  second 
through  the  JCS.  Air  Force  Major  Commands . . .  provide  their  input 
to  [the  Air  Staff];  Specified  Commands  through  the  JCS  as  well  "(DAF, 
1987:  14). 

In  terms  of  Air  Force  decision  making,  the  DOD  programming 
phase  is  probably  the  most  critical.  Programming  in  the  Air  Force  is 
accomplished  through  the  Program  Objective  Memorandum  (POM). 
The  POM  is  the  document  which  is  developed  and  coordinated 
throughout  the  Air  Force  to  match  available  funding  with  the  most 
critical  identified  needs.  The  key  product  of  this  phase  is  the  5-year 
defense  resource  proposal  (Five  Year  Defense  Program  -  FYDP)  which 
becomes  the  foundation  for  the  Air  Force  budget  submission  to  DOD 
(DAF,  1987:  9).  The  FYDP  documents  the  SECDEF  approved  programs 
for  the  DOD.  It  is  a  detailed  and  integrated  summary  of  resources, 
forces,  costs,  manpower,  procurement  and  construction  (DAF,  1987:  5). 

It  is  during  the  POM  coordination  process  that  many,  if  not  all, 
key  resource  allocation  decisions  are  made. 

The  programming  segment  is  the  first  point  in  the  PPBS 
process  where  fiscal  constraints  are  matched  against 
resource  requirements  and  it  is  likely  to  impact 
alternatives  selected  during  the  planning  segments  (DAF, 

1987:  17) 

A  critically  important  feature  of  the  programming  phase  of  the 
PPBS  is  the  coordination  that  is  accomplished.  The  POM  is  a 
corporate  Air  Force  program  proposal  and  is  based  upon  the  DG  (DAF, 
1987:  17).  The  coordination  of  the  Air  Force  program  proposal  during 
the  POM  deliberations  at  the  Air  Staff  are  the  final  adjustment  to 


program  funding  levels  prior  to  submission  to  OSD.  The  Air  Staff 
coordination  process  is  accomplished  through  the  Air  Force  Board 
structure.  "A  key  feature  of  the  AF  POM  development  process  is  the 
use  of  a  corporate  review  body  -  the  Air  Force  Board  Structure" 

(DAF,  1987:  18).  The  Board  Structure  'brings  together'  all  aspects  of 
the  Air  Force  POM  development  process,  provides  for  'openness' 
during  the  POM  development  process,  and  provides  recommendations 
to  the  Chief  of  Staff  of  the  Air  Force  and  to  the  Secretary  of  the  Air 
Force  (DAF,  1987:  18). 

The  actions  taken  during  POM  development  become  key 
determinants  of  the  life  or  death  of  programs.  No  matter  how  well 
stated  a  requirement  may  be,  or  how  attractive  a  technological 
opportunity  is,  without  funding  support  in  the  POM,  that  program 
will  not  succeed.  Within  the  acquisition  community,  funding  in  the 
POM  is  recognized  as  a  key  and  continuing  milestone  for  a  PM  to 
negotiate  during  a  program's  life  (ACSC,  1981:  27). 

The  POM  is  submitted  first  to  the  DOD,  then  to  the  Office  of 
Management  and  Budget  (0MB),  and  ultimately  to  the  Congress  as 
part  of  the  President's  budget  proposal.  The  budgeting  phase  of  the 
PPBS  involves  the  execution  of  Congressional  authorization  and 
appropriation  legislation.  The  budget  is  the  "...  planned  program  for 
a  fiscal  period  in  terms  of  estimated  costs,  obligations,  and 
expenditures"  (DAF,  1987:  47).  The  budget  is  the  foundation  upon 
which  the  resource  management  control  systems  of  the  Air  Force  are 
based.  Resource  management  information  focuses  upon  outputs  and 
resources  used,  the  managers  responsible  for  mission  accomplishment, 


actual  performance  in  relation  to  planned  performance  (i.e.  budgeted 
requirements),  expense  operating  budgets  and  accounting  systems  to 
enhance  management  control,  and  working  capital  techniques  to 
assist  in  accounting  for  inventory  (AFIT,  1987;  76). 

The  role  of  the  Program  Manager  in  the  budget  sub-process  is 
considerable.  The  POM  is  'built  from  the  bottom  up'  (DAF,  1987;  25). 
Individual  programs  are  documented  in  Program  Decision  Packages 
(PDP)  and  assigned  a  single  Program  Element  Monitor  (PEM)  who  is 
the  Air  Staff  OPR  for  that  program  (DAF,  1987;  23).  The  PEM, 
however,  gets  the  necessary  information  from  the  field.  In  the 
acquisition  arena  this  means  the  PM.  The  PM  is  responsible  for 
"...  acquiring  and  fielding  a  system  that  meets  the  approved  mission 
need  and  achieves  established  cost,  schedule,  readiness,  and 
affordability  objectives  (DOD,  1986.  13).  Under  the  tenants  of 
participatory  management  and  the  bottoms  up  approach  to  building 
the  Air  Force  POM,  the  PM  is  specifically  responsible  for  making 
"...  timely  planning  and  programming  inputs ...  to  identify  funding 
requirements  for  the  acquisition  program  (DAF,  1986:  6)  The  budget 
sub-process  is  summarized  in  Figure  3. 

The  Acquisition  Management  Sub-Process.  The  DOD 

directive  that  guides  weapon  system  acquisition  activities, 
Department  of  Defense  Directive  5000.1  -  Major  System  Acquisitions, 

states  that  it  "...  is  the  policy  of  the  Department  of  Defense  to 
ensure  that  DOD  acquisition  of  major  defense  systems  is  carried  out 
efficiently  and  effectively  to  achieve  the  operational  objectives  of  the 


Figure  3 

The  Budget  Sub-Process 
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U  S.  Armed  Forces  in  their  support  of  national  policies  and  objectives" 
(DOD,  1986:1). 

The  process  is  structured  to  accomplish  those  goals  by  having 
each  program  progress  through  a  series  of  four  key  decision  points, 
or  milestones,  at  which  specific  items  are  addressed.  The  key 
decisions  that  are  made  at  these  milestones  are  primarily  retained  " 
...  at  the  level  of  agency  head"  (0MB,  1976:  6-7).  In  the  DOD,  this  is 
the  Secretary  of  Defense  (SECDEF).  As  a  program  passes  each 
milestone,  programmatic  and  technical  risks  are  judged  to  have  been 
reduced  sufficiently  to  warrant  the  commitment  of  additional 
resources  toward  the  completion  of  the  program.  The  resources 
concerned  are  primarily  funding  and  manpower  although  a  strong 
case  could  be  made  that,  implicitly,  time  is  also  very  important. 

The  four  key  milestones  are  defined  below  (DOD,  1986:  12-13). 

As  shown  in  Figure  4,  each  decision  results  in  incrementally  more 
resources  committed  to  the  program  in  response  to  decreasing 
program  risk. 

Milestone  0,  Mission  Need  Determination, 
complements  the  requirements  and  budget 
processes  discussed  above.  At  this  milestone,  a 
validated  requirement  is  combined  with  POM 
support  resulting  in  program  initiation.  The 
program  effort  immediately  after  this  decision 
addresses  conceptual  investigation  of  alternative 
methods  to  meet  the  operational  need  identified  in 
the  SON. 


Incrementally  More  Resources  Committed 


Figure  4 

The  Acquisition  Management  Sub-Process 


Decreasing  Technical  Risk 


Milestone  I,  Requirement  Validation,  takes  the 
results  of  the  conceptual  studies  and  determines  the 
best  direction  in  which  to  proceed.  For  major 
programs,  this  decision  is  made  by  the  Secretary  of 
Defense  (SECDEF)  after  review  of  the  program  by 
the  Defense  System  Acquisition  Review  Council 
(DSARC).  After  the  Milestone  I  decision,  the 
program  begins  the  demonstration  and  validation 
phase.  It  is  during  this  phase  that  the  Air  Force 
has  the  first  opportunity  to  test  methods  of 
satisfying  the  operational  need  that  originally 
triggered  the  program. 

At  Milestone  II,  Program  Go-Ahead,  the  DSARC 
again  reviews  the  program  for  adequate  progress. 

If  the  operational  need  is  still  current  and  if  the 
risks  continue  to  be  acceptable,  the  program  is 
approved  for  Full-Scale  Development  go-ahead  This 
decision,  however,  is  constrained  to  a  specific  level 
by  the  funding  that  has  been  made  available  by 
the  budget  process. 

The  decision  to  proceed  into  production  is  made  at 
Milestone  III,  Production/Deployment  Go-Ahead. 

Instead  of  a  DSARC  review,  the  authority  for  this 
decision  has  been  delegated  to  the  Secretary  of  the 
Air  Force  (SAF).  The  SAF  is  supported  by  the  Air 
Force  System  Acquisition  Review  Council  (AFSARC)  - 
a  body  analogous  to  the  DSARC  but  at  the  DAF 
level  (DOD,  1986:  12-13). 

The  role  of  the  PM  is  most  visible  in  the  acquisition 
management  process.  "The  Program  Manager  is  the  program's  chief 
executive,  chief  diplomat,  chief  doctor,  and  chief  ’flak-taker'  all  rolled 
into  one  person . . .  [and]  In  accordance  with  the  Air  Force  s  single 
manager  concept . . .  is . . .  assigned  ultimate  program  management 
responsibility"  (ACSC,  1981:  38).  The  specific  responsibilities  assigned  to 
the  PM  include  (DAF,  1986:  6): 


•  managing  the  assigned  acquisition  program  using 
the  assistance,  advice  and  recommendations  of 
participating  commands, 

•  making  management  decisions  within  the 
approved  program  (i.e.  as  documented  in  the 
Program  Management  Directive)  after  equally 
weighing  cost,  schedule,  performance,  supportability, 
training  requirements,  and  reliability  and 
maintainability; 

•  preparing  *  program  baseline  to  assess  and  to 
document  how  the  program  is  to  progress; 

•  ensuring  that  all  program  documentation  receive 
adequate  communication  and  coordination  with  all 
program  participants, 

•  assessing  the  program's  progress  against  its 
objectives,  constraints  and  thresholds;  and 

•  making  timely  planning  and  programming 
inputs. 

To  summarize,  the  acquisition  management  sub-process  is  a 
sequence  of  decisions  that  determine  whether  a  stated  need  and  a 
program's  progress  warrant  its  attendant  costs  and  risks.  The  PM, 
as  the  single  manager  responsible  for  all  technical  and  business 
decisions  (ACSC,  1981:  39),  is  the  key  link  in  the  Air  Force's  acquisition 
decision  making  chain. 

The  Contracting  Sub-Process.  The  progress  of  a  program 
from  one  milestone  to  the  next  depends  not  only  on  success  in  the 
budget  sub-process  (i.e.  funding  provided)  but  also  on  the  ability  of 
the  program  management  staff  to  accomplish  the  technical 
development  and  production  efforts  required.  The  four  phase 


contracting  sub-process  is  the  system  that  guides  the  technical 
conduct  of  a  program  (Westfall,  1987:  160). 

The  contracting  sub-process  can  occur  many  times  during  the 
life  of  a  single  weapon  system  acquisition.  The  first  contracting 
phase  is  Requirements  Definition.  Based  upon  the  results  of  decisions 
in  other  processes  (the  requirements  sub-process,  the  budget  sub¬ 
process,  or  the  acquisition  management  sub-process),  the  tasks 
required  to  be  accomplished  by  a  contractor  in  support  of  the 
program's  approved  objectives  are  laid  out  for  inclusion  in  a  Request 
for  Proposal  (RFP).  The  RFP  is  the  document  that  contractors  base 
their  proposals  on,  both  in  terms  of  their  technical  approach  and 
price.  The  contract  will  request  effort  consistent  with  the  validated 
SON,  at  a  level  within  the  established  funding  constraints  in  the 
approved  POM  and  provided  in  the  budget,  and  at  a  time  consistent 
with  the  goals  of  the  overall  weapon  system  acquisition  life-cycle 
(Westfall,  1987:  163-164). 

The  second  phase  is  Procurement  Planning.  It  is  this  time 
when  decisions  are  made  concerning  the  type  of  contract  anticipated 
(e.g.  fixed  price,  etc.)  and  possible  sources  to  be  solicited.  The  latter  is 
particularly  important  today  because  this  implies  the  degree  of 
competition  possible. 

After  contractor  proposals  are  received  by  the  program 
managers,  the  program  managers  evaluate  the  proposals  to 
determine  who  the  ultimate  winner  will  be.  This  phase  is  called 
Offer  Evaluation.  Within  this  phase  is  yet  another  process,  the 
Source  Selection  Process,  which  controls  the  evaluation  of  the 


contractor  proposals.  Each  proposal  is  evaluated  against  specific  and 
uniform  assessment  criteria.  These  assessment  criteria  can  include 
soundness  of  approach,  understanding  the  requirements,  compliance 
with  requirements,  past  performance  and  other  evaluation  criteria 
(e.g.  cost)  (DAF,  1984:  8,  19). 

In  major  weapon  system  acquisitions  cost  does  not  constitute 
the  only  discriminating  factor.  Given  the  complexity  of  modern 
weapon  systems,  and  the  unknowns  involved  in  their  development, 
the  technical  and  managerial  criteria  carry  equal  weight  with  price 
It  would  not  be  desirable  to  award  a  contract  to  the  lowest  bidder 
only  to  find  that  the  firm  is  unable  to  perform  satisfactorily  The 
dollar  sums  involved  are  simply  too  great  (Westfall,  1987:  164-165) 

The  Contract  Management  phase  follows  the  award  of  the 
contract  and  lasts  until  the  contract  is  completed.  This  may  not 
coincide  with  completion  of  the  entire  program.  Again,  many 
contracts  can  make  up  any  given  acquisition  program  The 
completion  of  one  is  often  just  the  beginning  of  another.  During  the 
Contract  Management  phase  the  contractor  is  monitored  to  ensure 
that  the  terms  of  the  contract  are  complied  with  and  that  necessary 
changes  are  implemented  (Westfall,  1987:  165-166).  The  contracting 
sub-process  is  summarized  in  Figure  5. 

Fox  points  out  that  in  the  contracting  process  the  roles  of  the 
PM  and  the  Procurement  Contracting  Officer  (PCO)  sometimes 
overlap.  The  tasks  normally  within  the  province  of  the  PCO  include 
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•  administration  of  contracts,  under  the 
supervision  of  the  PM; 

•  approval  of  subcontract  terms,  conditions,  and 
costs; 

•  preparation  of  field  analyses  of  contract-change 
proposals  and  their  impact  on  costs,  schedules,  and 
technical  performance;  and 

•  miscellaneous  duties,  including  interpretation  of 
contract  provisions,  review  and  analysis  of  reports, 
progress  surveillance,  and  across  the  board  support 
for  the  PM. 

The  Program  Manager's  prerogatives  during  the  contracting 
process  include  (Fox,  1974:  175): 

•  overall  technical  control,  including  responsibility 
for  decisions  affecting  design,  reliability,  and  trade¬ 
offs  in  design  because  of  cost  and  schedule 
considerations, 

•  approval  of  specifications, 

•  approval  of  plans  to  subcontract, 

•  technical  guidance  in  the  selection  of 
subcontractors;  and 

•  approval  of  contract-change  proposals  involving 
design  modifications,  additional  costs,  and  changes  in 
schedule. 

Although  only  the  PCO  retains  the  authority  to  obligate  the 
government  contractually,  the  role  of  the  PM  is  still  significant  in 
terms  of  overall  decision  making  duties  and  responsibilities  and  the 
requirements  for  information  generation  and  communication.  As 
illustrated  in  Figure  6,  there  is  a  track  of  documentation  through  the 
overall  weapon  system  acquisition  system.  This  track  of  program 
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documents  overlaps  the  four  sub-processes  discussed  above.  The  PM 
is  the  link  between  the  sub-processes  and  the  management 
continuity  from  the  early  phases  of  the  program  through  the  final 
termination  of  the  contractual  effort. 

The  key  documents  that  provide  this  track  are  the  Statement 
of  Operational  Need  (SON),  the  Program  Objective  Memorandum 
(POM),  the  Program  Management  Directive  (PMD),  the  Request  for 
Proposal  RFP)  and  the  contract.  The  SON  provides  the  organizational 
requirement,  the  POM  is  an  organizational  statement  of  prioritized 
program  funding,  the  PMD  is  is  organizational  direction  to  an  action 
officer  to  initiate  program  effort,  and  the  RFP  and  contract  are  the 
organization's  interface  with  industry. 

Recent  Environmental  Changes.  The  preceding  was  a 

necessarily  brief  review  of  the  major  components  of  the  weapon 
system  acquisition  process.  The  review  focussed  on  the  complexities 
of  decision  making  during  major  weapon  system  acquisitions,  the 
documentation  produced  and  the  role  of  the  program  manager  in  the 
sub-processes.  The  degree  of  overlap  between  the  different  sub¬ 
processes  and  their  inherent  structural  relationships  are  considerable. 
What  is  apparent  is  that  decision  making  within  the  Air  Force 
acquisition  community  is  interwoven  to  a  considerable  extent  and 
that  a  change  in  one  sub-process,  at  any  organizational  level,  can 
possibly  have  significant  and  unforeseen  ramifications  in  others. 

To  illustrate  these  possibilities,  four  specific  changes  in  the  Air 
Force  acquisition  environment  will  be  discussed.  The  intent  is  to 
show  that  while  the  specific  objectives  of  each  of  these  changes  can 
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be  concisely  stated,  the  side-effects  which  they  promulgate  might  not 
be  readily  apparent.  As  before,  the  focus  of  the  discussion  will  be 
the  acquisition  decision  making  role  of  the  PM. 

The  changes  to  be  discussed,  and  the  processes  they  impact, 
based  on  the  discussion  above,  are  shown  below  in  Figure  7.  The 
relative  impacts  on  the  sub-processes  for  each  of  the  environmental 
influences  were  subjectively  estimated  by  the  author  based  upon  the 
references  cited. 
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Environmental  Changes  &  Impacts 


Three  of  the  changes  are  legislation  passed  by  the  Congress  and 
signed  into  law  by  the  President.  The  last  is  the  result  of  a 
Presidential  Blue  Ribbon  Panel.  The  acquisition  management  process 
primarily  intended  to  be  influenced  by  the  change  is  identified  as 
primary  while  others  affected  are  identified  as  secondary 


Balanced  Budget  and  Emergency  Deficit  Control  Act.  The 
Graham-Rudman-Hollings  legislation  is  perhaps  the  most 
controversial  in  recent  years.  Officially  titled  the  Balanced  Budget 
and  Emergency  Deficit  Control  Act  (P.L.  99-177),  it  is  intended  to 
remedy  problems  in  the  Congressional  budget  process  as  defined  in 
the  existing  law,  the  Congressional  Budget  and  Impoundment  Control 
Act  of  1974  (P.L.  93-344). 

Although  the  budget  formulation  procedures  enacted  in 
Graham-Rudman-Hollings  are  very  similar  to  those  in  effect  since 
1974,  there  are  changes.  The  two  most  significant  changes  are 

the  setting  of  specific  deficit  targets  -  ‘maximum  deficit 
amounts'  -  for  fiscal  years  1986  through  1991  which  the 
President  and  Congress  must  follow,  and  a  new 
enforcement  mechanism  -  'sequestration1  -  that 
automatically  cuts  federal  spending  by  whatever  amount 
is  needed  to  reach  the  deficit  target  for  the  coming  year 
when  the  President  and  Congress  are  unable  or  unwilling 
to  agree  on  a  deficit  reduction  plan  of  their  own" 

(Collender,  1986:  15) 

While  the  setting  of  maximum  deficit  amounts  is  perhaps 
admirable  and  even  necessary,  it  is  the  provision  for  sequestration 
and  what  it  entails  with  respect  to  decision  making  that  is  of 
interest  here  Secretary  of  Defense  Weinberger  has  likened 
sequestration  to  an  indiscriminate  club  that  Congress  will  swing  to 
cut  the  budget  (Weinberger,  1986:  21).  Sequestration  is  a  process  that 
will  occur  automatically  without  further  Congressional  debate  or 
Presidential  discretionary  action.  The  law  prescribes  what  will 
happen  and  when 


At  the  end  of  last  fiscal  year  (FY  86)  the  sequestration  process 
was  invoked.  Although  this  specific  procedure  within  Graham- 
Rudman-Hollings  has  been  challenged  and  found  unconstitutional  on 
the  grounds  that  it  is  an  unconstitutional  delegation  of  authority  to 
the  Comptroller  General  (a  part  of  the  Executive  Branch),  it  was 
exercised  in  1986. 

It  is  the  fact  that  sequestration  is  an  'indiscriminate  club’ 
which  requires  its  discussion  here  in  the  context  of  acquisition  PM 
decision  making.  The  law  provides  that  all  eligible  programs,  defense 
and  non-defense,  must  equally  share  the  burden  of  the  budget 
reductions  required  to  meet  the  maximum  allowable  deficits  as 
written  in  the  legislation.  By  definition,  the  process  does  not  allow 
consideration  of  priorities  among  competing  programs.  Regardless  of 
the  apparent  irrationality  of  a  cut  to  a  particular  program's  budget, 
either  in  economic  terms  or  its  possible  effect  on  national  security, 
without  decisive  action  on  the  part  of  the  Congress  and  the 
President,  the  cut  will  take  place. 

Consider  a  program  such  as  the  available  inventory  of 
munitions,  whose  health  is  critical  to  the  nation's  warfighting  ability. 
"The  progress  achieved  [in  recent  years]  in  the  inventory  levels  of 
our  munitions  is  truly  a  success  story.  But,  these  are  the  very 
types  of  successes  that  will  be  the  first  to  turn  into  shortages  under 
successive  Graham-Rudman  sequestrations,  since  there  is  no 
possibility,  under  the  law,  of  choosing  priorities"  (Weinberger,  1986: 

19). 


This  perturbation  to  the  setting  of  priorities,  and  subsequent 
acting  in  accordance  with  those  priorities,  is  the  critical  effect  of  the 
Balanced  Budget  and  Emergency  Deficit  Control  Act  upon  Air  Force 
acquisition  program  management  decision  making.  Its  effects  can 
trigger  repercussions  through  the  budget  process  as  the  Air  Force 
attempts  to  adjust  to  the  reductions  mandated  in  the  law.  Although 
the  budget  process  will  be  the  primary  location  of  this  activity,  other 
processes  will  be  effected  too. 

Since  the  ability  to  act  effectively  upon  established  priorities 
will  be  reduced  or  completely  removed,  the  process  of  determining 
requirements  must  now  take  that  possibility  into  account.  There  is 
already  a  tendency  to  propose  'requirements’  only  if  strong  'budget' 
support  is  considered  likely.  Requirements  might  be  withheld 
prematurely  because  of  perceived  budget  risk  before  they  had  a 
sufficient  review  in  the  SON  validation  cycle.  Operational 
requirement  determination  could  become  a  follow-on  to  the  budget 
process  instead  of  the  reverse  (Meehan,  1985:  29-39). 

Even  after  a  program  has  been  initiated,  the  effects  of 
Graham-Rudman  will  still  be  felt.  A  program  in  the  production 
phase,  for  example,  could  have  its  budget  cut,  requiring  among  other 
things,  a  reduction  in  yearly  buy  quantities.  The  mistaken  opinion 
of  some  is  that  this  results  in  cost  savings  where,  in  fact,  the 
opposite  is  true.  The  contract  change  required  to  adjust  the 
quantity  will  cost  the  Air  Force  program  extra  funds  in  the  year  it 
is  enacted  and  the  unit  price  of  the  items  being  purchased  will  also 
rise  (Gansler,  1980:  220-224). 
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The  Competition  in  Contracting  Act.  The  Competition  in 
Contracting  Act  was  passed  by  Congress  and  signed  into  law  by  the 
President  in  1985.  The  act  embodies  the  prevailing  mood  in  the 
Congress  that  competition  in  government  contracting  will  result  in 
lower  prices  and  less  corruption.  This  may  not  be  so.  Some  are 
concerned  that  "...  competition  is  not  axiomatic  with  reduced 
program  cost"  (Grosen  and  Augusta,  1986:  33).  Issues  such  as  dual 
sourcing  and  effects  on  learning  curves,  unit  prices  and  total 
program  costs  have  yet  to  be  fully  addressed  (Grosen  and  Augusta, 
1986:  36). 

These  concerns  are  not  the  primary  issue  here,  however.  For 
whether  there  are  cost  savings  to  be  reaped  from  competition  or  not, 
the  new  emphasis  on  competition  is  already  affecting  the  Air  Force 
contracting  process. 

In  the  procurement  planning  phase  of  the  contracting  process, 
competition  is  now  the  key  goal  when  laying  out  the  procurement 
plan.  Air  Force  contracting  personnel  and  program  managers  must 
take  all  measures  possible  to  achieve  'full  and  open  competition*. 
Anything  less  must  be  either  explained  and/or  justified. 

This  requirement  impacts  the  acquisition  management  sub¬ 
process  as  well.  A  program  manager  must  be  careful  how  the 
contracts  are  arranged  in  terms  of  budgetary  timing  and  the 
technical  phase  of  the  program.  A  contractor  whose  contract  expires 
may  not  be  able  to  simply  renegotiate  and  continue.  The  balance  of 
the  effort  might  be  competed  and  the  contract  awarded  to  another 
firm. 


Although  this  sounds  good  on  the  surface,  from  a  program  cost 
view  point,  the  new  potentially  less  expensive  contractor  must  now 
be  brought  'up-to-speed'.  There  will  be  inevitable 
miscommunications  and  false  starts.  These  costs,  however,  are  very 
difficult  to  document. 

Department  of  Defense  Reorganization  Act.  There  has 
been  concern  for  some  time  about  the  Pentagon's  ability  to  plan.  As 
early  as  1979,  studies  on  the  organization  of  the  DOD  "...  either 
explicitly  or  implicitly  addressed  the  capability  of  the  Department  of 
Defense  to  plan"  (Barrett,  1981;  112). 

On  1  October  1986,  the  President  signed  the  Goldwater-Nichols 
Department  of  Defense  Reorganization  Act.  The  act  goes  beyond 
simply  reforming  the  JCS  and  instead  attempts  to  improve  multi¬ 
service  planning,  coordination  and  warfighting.  More  power  will  be 
given  to  the  Commanders  in  Chief  of  the  Unified  and  Specified 
Commands.  This  includes  not  only  operational  authority,  but  also  a 
greater  say  in  programming  decisions. 

The  JCS  staff  will  be  strengthened  as  will  the  Chairman  of  the 
Joint  Chiefs.  These  actions  are  intended  to  improve  planning  at  the 
joint  level  and  provide  better  'military  advice’  from  the  top  military 
spokesman  -  the  Chairman  of  the  JCS. 

The  military  departments  (Air  Force,  Army  and  Navy)  are  also 
reorganized.  Specifically, 

. . .  the  sole  responsibility  for  acquisition,  auditing, 
comptroller  functions,  information  management,  inspector 
general  duties,  legislative  affairs,  and  public  affairs  will 
be  consolidated  in  each  Service  secretariat.  The  military 


headquarters  staffs  are  barred  from  establishing  or 
designating  any  office  within  their  headquarters  to 
conduct  any  of  those  functions  (Ganley,  1986:  25) 

Although  the  primary  focus  of  these  three  actions  is  on  the 
planning  function,  there  will  undoubtedly  be  significant  impacts  on 
both  the  budgeting  process  and  the  acquisition  management  sub¬ 
process.  The  PPBS  will  likely  see  a  shift  in  power  toward  field 
commanders.  Whether  or  not  this  will  affect  decision  making  process 
results  is  unknown,  but  it  will  probably  result  in  qualitatively 
different  programming  decisions  (Ganley,  1986:  24). 

With  a  single  Air  Force  acquisition  staff  in  the  Pentagon  now, 
the  level  of  review  that  programs  receive  will  change.  An  intent 
was  to  reduce  the  levels  of  bureaucracy  that  a  program  manager 
had  to  go  through  in  order  to  gain  approval  for  a  decision.  As  above, 
whether  or  not  that  will  actually  be  the  case  remains  to  be  seen 
(Ganley,  1986:  25). 

The  Packard  Commission  Report.  The  President's  Blue 
Ribbon  Commission  on  Defense  Management  (the  Packard 
Commission)  submitted  its  report  to  the  President  and  the  Secretary 
of  Defense  on  30  June  1986.  It  contained  recommendations  for 
improving  the  management  of  the  DOD  in  four  areas:  National 
Security  Planning  and  Budgeting,  Military  Organization  and  Control, 
Acquisition  Organization  and  Procedures,  and  Government-Industry 
Accountability. 

Many  of  the  recommendations  presented  in  the  report  are 
being  implemented.  For  example,  the  Goldwater-Nichols  Act  of  1986 
enacted  most  of  the  Commission's  Military  Organization  and 


Command  recommendations.  Specifically,  the  role  of  the  JCS  in  both 
planning  and  force  employment  was  strengthened. 

The  importance  of  the  report  to  this  discussion  is  that  it 
demonstrates  that  a  systematic  and  comprehensive  and  integrated 
review  of  decision  making  elements  can  be  accomplished.  This  study 
shares  that  goal.  As  shown  in  Figure  7,  the  Commission’s 
recommendations  touched  all  four  of  the  weapon  system  acquisition 
decision  making  sub-processes  discussed  earlier  in  this  section. 

Without  Congressional  action,  however,  many  of  the  more 
significant  recommendations  can  not  be  implemented.  Deputy 
Secretary  of  Defense  Taft  stated  clearly  that 

If  the  commission's  recommendations  are  to  be  fully 
realized,  the  Congress  must  make  changes,  not  by 
enacting  legislation  aimed  at  the  executive  branch  but  by 
altering  the  way  it  conducts  its  responsibilities  for 
national  security.  Among  changes  needed  are:  reduction 
of  the  line  item  decision-making  by  Congress,  adoption  of 
biennial  budgets,  and  limitation  of  defense  oversight  to 
the  committees  properly  charged  with  that  responsibility 
(Taft,  1986:  23). 

There  are  many  decisions  made  within  the  Air  Force,  DOD,  and 
Congress,  during  the  acquisition  of  a  new  weapon  system.  As 
discussed  above,  these  decisions  are  interrelated  through  a  network 
of  four  basic  sub- processes.  Changes  aimed  at  any  one  managerial 
level  decision  may  cause  unforeseen  changes  or  complications  in 
another  managerial  level  decision. 

The  acquisition  Program  Manager  is  the  constant  organizational 
focal  point  throughout  the  four  sub-processes.  It  is  also  the  PM  that 
is  the  focus  of  organizational  actions  taken  in  response  to  the  four 


recent  environmental  changes  discussed.  The  ability  of  the  PM  to 
respond  to  environmental  influences  while  at  the  same  time  working 
within  four  distinct  acquisition  system  sub-process  is  key  to  the 
efficient  and  effective  conduct  of  the  program. 

Management  Theory 

Management  has  been  defined  as  a  process  undertaken  by 
individuals  to  coordinate  the  work  of  others  to  achieve  results  that 
would  not  have  been  achieved  by  any  of  the  individuals  acting  alone 
Managers  are  the  individuals  responsible  for  achieving  these  results 
through  the  specialized  efforts  of  others  (Donnelly,  et  al,  1984:  3). 

Management  can  be  further  defined  in  terms  of  the  functions 
the  manager  is  responsible  for  accomplishing.  The  five  functions 
originally  described  by  Henri  Fayol  in  1916  are  still  used  today.  The 
functions  of  management  are  planning,  organizing,  coordinating, 
commanding  (or  directing),  and  controlling.  The  focus  of  the  Fayol 
school  of  management  represented  a  shift  from  the  workshop 
orientation  of  Frederick  Taylor  to  an  entire  organization  perspective 
(Hellriegel  8.  Slocum,  1974:  58-59). 

Organizational  Structure.  As  discussed  above,  the  structure  of 
Air  Force  acquisition  decision  making  is  complex  organizationally. 
Complex  organizational  structures  come  in  several  types,  from  the 
t!  .eories  of  Classical  management  to  the  behaviorists  (Miner,  1982, 
viii).  The  theory  of  bureaucracy,  general  systems  theory  and  socio- 
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technical  systems  theory  though  are  of  primary  interest  to  this 
discussion. 

The  bureaucratic  model  is  the  oldest  and  most  prevalent  model 
of  management.  Though  not  a  monolithic  concept,  the  following 
characteristics,  while  varying  in  degree  from  application  to 
application,  capture  ’the  essence'  of  the  bureaucratic  model  of 
organization  (Harrison,  1978:  205-210). 

•  a  division  of  labor  based  on  functional 
specialization, 

•  a  well-defined  hierarchy  of  authority, 

•  a  system  of  rules  covering  the  rights  and  duties 
of  positional  incumbents, 

•  a  system  of  procedures  for  dealing  with  work 
situations, 

•  impersonality  of  interpersonal  relations,  and 

•  promotion  and  selection  for  employment  on  the 
basis  of  technical  competence  (Harrison,  1978:  207) 

The  chief  advantage  of  the  bureaucratic  model  is  its  technical 
efficiency  which  emphasizes  precision,  speed,  expert  control, 
continuity,  discretion  and  optimal  returns  on  investment  (Harrison, 
1978:  210).  Disadvantages  discussed  in  the  literature  include:  adverse 
impacts  on  motivation,  a  tendency  toward  goal-displacement  and 
self-service,  the  creation  of  elitism,  and  a  loss  of  control  over  the 
operations  of  the  organization  (Harrison,  1978.  211-212). 

The  systems  model  is  a 

. . .  holistic  concept  in  that  it  views  the  organization  as  a 

totality  composed  of  interrelated  processes  and  functions. 


functions.  It  is  also  a  global  concept  in  that  it  views  the 
organization  as  operating  in  a  larger  environmental 
universe.  Finally,  it  is  a  synergistic  concept  in  that  it 
assumes  the  optimal  integration  of  the  elements  that 
comprise  the  system  (Harrison,  1978:  267).  [Emphasis  in 
original] 

Systems  theory  is  based  to  a  large  extent  on  the  work  of  von 
Bertalanffy  and  Boulding  and  their  efforts  to  define  a  General 
Systems  Theory  that  would  apply  to  any  discipline.  In  this  regard 
General  Systems  Theory  is  the  "conceptual  underpinning"  of  the 
systems  model  (Harrison,  1978:  271).  Systems  theory  has  been 
applied  to  several  activities  within  organizations:  the  total  process  of 
management,  managerial  decision  making,  and  information 
processing  (Harrison,  1978:  268).  The  following  is  offered  as  a 
comprehensive  definition  of  a  management  system. 

A  management  system  can  be  defined  as  that  subsystem 
of  the  organization  whose  components  consist  of  a  subset 
of  individuals  (man  to  man)  whose  duties  are  to  receive 
certain  organizational  problems  (inputs)  and  thereupon 
to  execute  a  set  of  activities  (process)  which  will  produce 
organizational  solutions  (outputs)  for  either  increasing 
the  value  or  return  of  the  total  organizational  activity 
(satisficing)  or  for  optimizing  some  function  of  the  total 
organizational  inputs  and  outputs  (Young,  1966:  15) 

[Emphasis  added] 

Management  systems  can  also  be  defined  as  "...  a  number  of 
components  or  subsystems  that  can  be  related  to  a  plan  setting  forth 
the  objectives  of  the  organization  (Harrison,  1978:  270).  Relating  back 
to  an  overall  definition  of  management,  systems  theory  states  that: 

Managers  are  needed  to  convert  the  disorganized 
resources  of  men,  machines,  and  money  into  a  useful  and 
effective  enterprise.  Essentially,  management  is  the 


process  whereby  these  unrelated  resources  are  integrated 
into  a  total  system  for  objective  accomplishment 
(Johnson,  et  al,  1973:  15). 

In  discussions  of  systems  theory,  the  concepts  of  closed  and 
open  systems  must  be  introduced,  A  closed  system  is  essentially  self 
contained,  with  all  its  resources  present  within  it.  A  closed  system 
neither  impacts  its  environment  nor  is  impacted  by  its  environment 
Open  systems,  however, 

. . .  import  resources  from  the  environment,  transform 
them  into  some  useful  output,  and  export  the  output  into 
the  environment.  This  input-transformation  [process]- 
output  cycle  undergoes  continuous  iteration  (Schoderbek, 
et  al,  1985:  45)  [ , . .  ]  added 

Figure  8  summarizes  the  systems  theory  of  management. 
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Figure  8 

The  Open  System  Model 


Within  an  organizational  construct  defined  as  an  open  system, 
Emery  and  Trist  described  socio technical  systems  as  "...  the 
technological  component  in  converting  inputs  into  outputs"  (Harrison, 
1978:  327).  As  such,  the  sociotechnical  systems  functions  as  a 
boundary  spanner  between  the  organization,  its  members,  and  its 
environment  (Harrison,  1978:  328).  This  relationship  is  summarized 
in  Figure  9. 


Figure  9 

The  Socio-Technical  System  Model 


The  following  are  definitions  (Harrison,  1978:  328-329)  applicable 
to  Figure  9: 

Systemic  Inputs  - 

•  Human  inputs  including  personality,  skills, 
values,  intelligence,  expectations  and  needs, 


•  Technical  inputs  including  techniques, 
knowledge,  information,  materials,  and  facilitating 
services, 

•  Managerial  inputs  including  objectives,  plans, 
decisions,  authority,  communication,  and  controls, 
and 

•  Environmental  inputs  including  societal, 
political,  ecological,  cultural,  legal,  and  religious. 

Sociotechmcal  Behavior  (Social  elements)  - 

•  Activities  or  the  things  people  do,  such  as 
walking,  talking,  sitting,  working,  and  thinking, 

•  Interactions  or  actions  in  relation  to  other 
people,  such  as  helping,  sharing,  and  meeting;  and 

•  Sentiments  or  the  feelings  people  develop  about 
what  they  are  doing,  what  is  being  done  to  them, 
and  the  people  with  whom  they  are  associated 

Sociotechmcal  Behavior  (Technical  elements)  - 

•  Technology  which  is  the  current  level  of 
knowledge  or  the  state-of-the-art  with  regard  to 
the  processes  and  operations  required  to  transform 
systemic  inputs  into  systemic  outputs; 

•  Processes  which  include  the  various  sequences  or 
groups  of  operations  required  to  apply  the 
technology  in  the  various  stages  of  the  total 
transformation  process;  and 

•  Operations  which  entail  particular  acts  or 
applications  of  the  individual  processes  through 
which  the  systemic  inputs  are  progressively 
transformed  by  stages  into  systemic  outputs 


Systemic  Outputs  - 

•  Production  which  is  a  tangible  measure  of 
organizational  accomplishment  relative  to  the 
resources  committed  to  obtaining  it, 

•  Innovation  which  is  technological  change 
involving  new  products  or  new  uses  for  existing 
products; 

•  Satisfaction  which  is  a  measure  of  fulfillment  of 
human  needs,  and 

•  Development  which  is  a  measure  of  the 
organization  as  a  more  viable  entity  or  the 
individual  as  a  more  accomplished  performer. 

Within  sociotechnical  theory,  technology  is  defined  as  the 
systematic  application  of  organized  knowledge  to  practical  tasks. 
Technology  is  "...  inextricably  interrelated  with  the  other  systems 
that  make  up  the  total  organization  . . .  and  it  "...  exerts  a 
substantial  influence  on  the  formal  organization  structure,  the 
system  of  management,  and  the  social  system  of  a  given  formal 
organization”  (Harrison,  1978:  338). 

Systems  can  be  divided  and  sub-divided  into  subsystems  and 
subsubsystems.  This  process  is  called  'nesting’  (Schoderbek,  et  al, 
1985:  51).  Several  propositions  apply  to  the  nesting  of  systems. 

1.  A  system  is  always  made  up  of  other  systems. 

2.  Given  a  system,  another  can  always  be  found 
that  comprises  it. 

3.  Given  two  systems,  the  one  comprising  the 
other  can  be  called  a  higher  level  system  and  the 
system  comprised  by  the  higher  level  system  can 
be  called  the  lower  level  system. 


4.  A  hierarchy  of  systems  exists  whereby  lower 
level  systems  are  part  of  higher  level  systems. 

5  The  lower  level  systems  are  m  turn  made  up  of 
other  systems  and  can  be  considered  the  higher 
level  system  for  the  lower  level  systems  to  be 
found  in  it  (Van  Gigch,  1974:  44). 

The  preceding  discussion  of  organizational  structure  illustrates 
the  basic  theoretical  foundations  available  for  the  development  of  an 
normative  organizational  decision  making  framework.  The 
organizational  situation  faced  by  the  acquisition  Program  Manager  is 
approximated  by  the  bureaucratic  model.  This  is  true  particularly 
with  respect  to  the  authority  hierarchy  and  the  formalization  of 
procedures. 

To  make  this  approximation  closer  to  the  total  reality  faced  by 
the  acquisition  Program  Manager,  however,  the  open  systems  theory 
model  and  the  socio-technical  model  were  introduced.  The  open 
systems  theory  model  provides  for  a  way  of  effectively 
characterizing  the  flow  of  management  through  a  three  step 
paradigm:  inputs,  processes,  and  outputs.  Such  a  perspective  is 
valuable  for  the  PM  and  the  tasks  of  relating  the  four  different  sub¬ 
processes  to  the  overall  objectives  of  the  program 

The  socio-technical  model  provides  additional  richness  in  that  it 
expands  the  definitions  of  the  inputs  and  outputs  while  focusing  upon 
processes  as  being  comprised  of  many  facets,  both  formal  and 
informal.  In  the  leadership  and  communication  roles,  the  ability  of 
the  PM  to  consider  to  entire  systemic  situation  increases  the 
likelihood  of  effective  management  action. 


Organizational  Effectiveness.  Managers,  however,  cannot 

discharge  their  functional  responsibilities  without  using  other 

important  skills.  The  ability  to  make  effective  decisions  is  one  of 

these  skills.  Peter  F.  Drucker  stated  that  only  managers,  and  high 

level  executives  in  particular,  make  decisions.  They  may  do  other 

things  as  well,  but  the  "...  first  managerial  skill  is  ...  the  making  of 

effective  decisions"  (Drucker,  1974;  465). 

Drucker  defines  a  decision  as  a  choice  between  alternatives.  He 

goes  on  to  define  an  effective  decision  as  one  that  includes  a 

commitment  of  action  and  results  (Drucker,  1974;  476).  A  still  more 

rigorous  definition  is  proposed  by  Ronald  A.  Howard.  Howard  and 

Matheson  define  a  decision  as 

.  .  .  an  irrevocable  allocation  of  resources,  in  the  sense 
that  it  would  take  additional  resources,  perhaps 
prohibitive  in  amount,  to  change  the 
allocation . . .  (Matheson  &  Howard,  1983;  23). 

Ebert  and  Mitchell  reviewed  the  definitions  of  decisions  and 

decision  making  in  several  disciplines.  They  explored  both  choice 

behavior  and  problem  solving  behavior  dimensions  and  arrived  at  an 

integrated  definition.  They  defined  organizational  decision  making  as 

. . .  the  choice  processes  that  occur  in  the  organizational 
setting  together  with  the  choice  related  behaviors  and 
the  processes  producing  them  that  are  directed  towards 
the  resolution  of  organizational  problems  to  reach 
organizational  goals  (Ebert  &  Mitchell,  1975;  12). 

The  Ebert  and  Mitchell  definition  emphasizes  the  organization. 

This  is  a  critical  factor  since  managers  do  not  make  effective  decisions 

in  a  vacuum.  As  Drucker  pointed  out,  an  effective  decision  requires 


a  commitment  to  action.  Since  managers  evoke  action  from  other 
people,  the  ability  of  the  manager  to  make  decisions  within  the 
context  of  an  organization  is  the  key  to  effectiveness. 

Managerial  effectiveness  is  defined  in  terms  of  resource 
utilization  in  relation  to  organizational  goal  attainment.  On  the 
other  hand,  managerial  efficiency  is  defined  in  terms  of  the 
proportion  of  total  organization  resources  that  contribute  to 
productivity  during  the  transformation  process.  (Certo,  1983:  12-13). 
Anthony  defines  efficiency  as  simply  the  ratio  of  outputs  to  inputs 
and  effectiveness  as  the  relationship  between  a  responsibility  center's 
output  and  its  objectives  (Anthony  8,  Young,  1984:  18-19)  In  terms  of 
systems  theory,  efficiency  deals  with  the  ratio  of  inputs  to  outputs 
while  effectiveness  deals  with  the  appropriateness  of  the  outputs 
with  respect  to  organizational  goals  and  objectives  and  the  ability  of 
the  transformation  process  to  produce  those  outputs. 

Seashore  has  stated  that  the  ways  in  which  organizations 
employ  their  informational  resources  underlie  a  model  of 
organizational  effectiveness  called  the  decision-process  model 
(Seashore,  1983:  60-61).  Decision-process  models  view  organizations  as 
information  processing  and  decision  making  entities.  Seashore's 
model  evaluates  these  against:  rational  standards  of  intrinsic 
goodness  of  decisions  made,  the  appropriateness  of  the  decision 
process,  and  the  impact  on  systemic  integrity  or  goal  attainment. 
Seashore  goes  on  to  say  that: 

. . .  the  issue  of  fit  of  decision  processes  to  the 
organization's  situation  is  crucial  and  difficult,  requiring 
differentiation  among  organizations  as  to  their  youth  or 


maturity,  whether  in  information-rich  or  information- 
poor  environments,  whether  processing  a  relatively  stable 
or  instable  goal  structure,  whether  embedded  in  a  simple 
or  a  complex  array  of  institutional  constituencies 
(Seashore,  1983:  61). 

Another  issue  to  be  discussed  is  a  suitable  measure  for 
effectiveness.  Anthony  points  out  that  in  profit-oriented 
organizations,  the  difference  between  revenues  and  expenses  is  a 
suitable  measure  of  effectiveness  in  most  cases.  Profit  as  a  measure 
of  effectiveness  has  the  following  advantages:  it  is  a  single  criterion 
that  can  be  used  in  decision  making  to  evaluate  alternatives,  it 
permits  quantitative  analysis  where  benefits  can  be  directly  related 
to  costs,  it  provides  a  single  measure  of  effectiveness,  it  permits 
decentralization,  and  facilitates  comparison  between  different 
organizational  elements  (Anthony  &  Young,  1984:  39-40).  For 
nonprofit  organizations  like  the  Air  Force  this  is  not  the  case.  In  fact 
"  .  .  .  in  a  nonprofit  organization  the  difference  between  revenue  and 
expense  says  nothing  about  effectiveness"  (Anthony  8.  Young,  1984: 
19-20). 

"A  nonprofit  organization  is  an  organization  whose  goal  is 
something  other  than  earning  a  profit  for  its  owners.  Usually  its 
goal  is  to  provide  services"  (Anthony  &  Young,  1984:  35)  Whereas  m 
profit-oriented  organizations  decisions  can  be  made  using  profit,  in 
dollars,  as  an  unambiguous  measure  of  desirability,  in  nonprofit 
organizations  this  criteria  is  lacking.  In  nonprofit  organizations 

. . .  decisions  made  by  management  are  intended  to  result 
in  the  best  possible  service  with  the  available  resources, 
success  is  measured  primarily  by  how  much  service  the 


organizations  provide  and  by  how  well  these  services  are 
rendered  (Anthony  &  Young,  1984:  35-36). 

For  the  Air  Force,  in  terms  of  its  mission,  measuring 

effectiveness  is  complicated  by  the  fact  that,  as  an  organization,  it  is 

part  of  the  political  sphere  as  opposed  to  the  economic  or  social 

sphere  (Harrison,  1978:549).  Also,  the  Air  Force  serves  more  than 

one  constituency.  In  a  given  decision  making  situation,  different 

groups  might  share  an  interest  in  the  results  while  having  diverse 

views  on  what  constitutes  success.  Finally, 

. . .  military  needs  tend  to  be  uncertain,  and  there  are  no 
generally  accepted  utilitarian  criteria  by  which  to  judge 
the  adequacy  of  military  organizations.  In  fact,  short  of 
actual  war,  there  is  no  direct  test  of  the  effectiveness  of 
military  organizations  (Harrison,  1978:  554). 

Given  this  scenario,  managers  require  substantial  inputs  from 

both  inside  and  outside  the  organization  to  help  rationalize 

organizational  objectives  and  to  satisfy  constituencies'  divergent 

measures  of  effectiveness.  The  key  sustaining  input  required  is 

information.  Without  quality  information  about  the  many  facets  of 

the  decision,  the  manager  may  come  up  with  the  right  answer  to 

the  wrong  question.  Information  is  the  most  important  ingredient  to 

effective  managerial  decision  making  (Drucker,  1974:  471). 

Drucker's  discussion  of  effective  management  decision  making 

draws  heavily  upon  his  observation  of  the  Japanese  approach  to 

making  'decisions'.  He  notes  that  the  Japanese,  however,  never 

really  make  decisions,  at  least  as  decisions  are  thought  of  in  Western 

society.  Instead,  the  Japanese  spend  considerable  time  gathering 


information  about  the  nature  of  the  problem,  whether  it  really 
exists,  what  the  alternatives  are,  what  pitfalls  exist,  etc. 

The  Japanese  process  is  focused  on  understanding  the 
problem.  The  desired  end  result  is  action  and  behavior 
on  the  part  of  people  ...  they  discipline  themselves  not  to 
commit  to  a  recommendation  until  they  have  fully 
defined  the  question  and  used  the  process  of  obtaining 
consensus  to  bring  out  the  full  range  of  alternatives 
(Drucker,  1974:  469-470). 

This  process  of  gaining  consensus  requires  the  transfer  of 
considerable  amounts  of  information  among  the  decision  making 
participants.  This  communications  task  in  the  Japanese  example 
occurs  during  factfinding  and  coordinating  meetings,  of  which  there 
may  be  many.  In  Western  organizations,  the  communications  task 
is  often  a  functional  responsibility  carried  out  in  accordance  with  the 
defined  procedures  within  the  formal  organizational  structure. 
However,  if  the  organization  is  faced  with  a  unique  problem  that 
will  require  a  complex  and  innovative  solution,  the  existing 
information  may  not  prove  suitable  to  the  challenge. 

For  the  acquisition  Program  Manager,  the  issue  of  program 
management  effectiveness  centers  on  the  ability  to  communicate 
information  within  the  organizational  context  so  as  to  efficiently 
accomplish  the  programs  goals  and  objectives.  Without  an 
unambiguous  measure  of  effectiveness  like  profit,  the  PM  must  rely 
on  other  organizational  indications  of  the  goodness  of  defined 
alternative  courses  of  action.  In  this  situation,  management  action 
must  rely  on  an  organizational  consensus  about  the  merit  of  the 
choice  to  be  made.  The  Program  Manager  must  ensure  that  the 


managerial  level  decisions  made  conform  to  overall  organizational 
expectations  while  at  the  same  time  satisfying  the  procedural 
requirements  of  the  four  acquisition  system  sub-processes. 


Decision  Analysis 

Many  decision  support  systems,  or  tools,  have  been  designed 
and  offered  as  the  means  to  overcome  the  problem  of  insufficient 
information.  These  tools  provide  specific  answers  to  problems  that 
can  be  easily  quantified  and  subjected  to  computer  analysis  Post 
World  War  II  operations  research  tools  (e  g.  optimization  techniques 
such  as  linear  programming)  were  effective  primarily  with 
operationally  repetitive  problems.  In  the  1950's,  the  field  of 
management  science  developed  to  address  higher  level  management 
problems.  These  efforts,  however,  tended  to  focus  upon  problems 
that  could  be  solved  with  elegant  mathematics  (Matheson  &  Howard, 
1983.  21-22).  These  mathematical  models  did  not  provide  the  decision 
maker  with  information  about  the  problem  itself  -  they  provided 
little  insight  into  the  complex  nature  of  the  decision.  Also,  they  did 
not  explicitly  address  the  problem  of  intra-organizational  and  extra- 
organizational  communication. 

Optimization  techniques  and  elegant  mathematics  also  required 
simplifying  assumptions  be  made  concerning  both  the  nature  of  the 
decision  and  the  operational  characteristics  of  the  system  under 
examination.  Three  significant  assumptions  were  that  an  objective 
function  could  be  formed  containing  a  single  attribute  for  evaluation, 


that  outcomes  were  not  subject  to  uncertainty  (de  Neuville,  1971:  87) 
and  that  decision  makers  maximized  value  or  utility  (Simon,  1959- 
230). 

The  same  was  true  of  computer  simulations.  These  highly 
complex  computer  programs  included  many  implicit  and  subtle 
assumptions  that  affected  the  data  they  produced.  As  decision 
information  was  presented  to  groups  further  and  further  removed 
from  the  raw  simulation  data,  upon  which  the  decisions  were 
originally  based,  the  clarity  of  the  intended  message,  in  terms  of 
assumptions,  limitations,  and  risks,  was  diminished. 

The  assumptions  employed  in  the  tools  of  management  science 
are  necessary  because  the  real  world  is  complex  and  dynamic. 
Herbert  Simon  stated  that  even  with  these  powerful  new  tools, 
"most  real-life  choices  still  lie  beyond  the  reach  of  maximizing 
techniques  -  unless  the  situations  are  heroically  simplified  by  drastic 
approximations  ”  (Simon,  1959;  259)  The  real  world  and  its 
problems  are  not  simple  and  straight  forward.  They  are  messy. 
According  to  Howard  Raiffa,  there  is  an  art  rather  than  a  science  to 
analyzing  real  world  problems  (Raiffa,  1968:  239).  The  art’  required 
is  the  understanding  of  the  individual  decision  maker's  perceptions  of 
the  environment  and  how  he  or  she  manages  the  process  of  choice. 

The  principles  and  techniques  of  decision  analysis  offer 
the  opportunity  to  deal  with  these  problems.  Decision 
analysis  includes  procedures  and  methodology  for 
assessing  the  real  nature  of  a  situation  in  which  a 
decision  might  be  made,  for  capturing  the  essence  of  that 
situation  in  a  formal  but  transparent  manner,  for 
formally  'solving'  the  decision  problem,  and  for  providing 


insight  and  motivation  to  the  decision-makers  and 
implemented  (Howard  &  Matheson,  1983:  viii).  [Emphasis 
added] 

To  allow  decision  analysis  to  be  used  in  the  development  of  a 
normative  organizational  decision  making  framework,  however,  it  is 
necessary  to  discuss  decision  analysis  principles  and  techniques  with 
respect  to  the  changed  decision  making  environment  found  in  large 
organizations.  Conventional  decision  analysis  was  discussed  in  the 
literature  primarily  in  terms  of  solitary  decision  makers  making 
singular  decisions.  As  discussed  above,  however,  in  large 
organizational  decision  making  processes,  decisions  can  be 
accomplished  through  a  series  of  sequential  lower  level  decisions  each 
of  which  can  be  decided  by  one  manager  or  several.  It  is  also 
desirable  to  ensure  that  all  the  assumptions  underlying  decision 
analysis  methodology  are  understood  and  consistent  with  the 
organizational  decision  making  environment. 

The  basic  approach  to  decision  analysis  decision  making  is  a  five 
step  decision  analysis  cycle  used  by  the  Strategic  Decisions  Group,  a 
well  known  practitioner  of  decision  analysis  (Matheson  8.  Howard, 
1983:  17-55).  The  baseline  process  is  explained  below. 

Step  [:  Decision  basis  development.  The  problem  and  decision 

required  are  defined  in  terms  of  three  elements  of  the  basis:  the 
alternatives  to  be  considered,  the  information  required,  and  the 
preferences  of  the  decision  maker  .  The  decision  basis  is  the 
quantitative  specification  of  the  problem.  The  two  essential  steps  in 
decision  analysis  are  the  development  and  evaluation  of  this  decision 
basis  (Howard,  1983:  11). 


Step  2:  Deterministic  structuring.  "The  deterministic  phase  is 
essentially  a  systems  analysis  of  the  problem"  (Matheson  and 
Howard,  1968:  27).  Within  this  phase  there  are  two  areas  of 
activity:  modeling  and  analysis.  The  modeling  area  consists  of  the 
following: 

1.  Bound  the  decision  -  specify  exactly  what 
decision  is  to  be  made.  If  alternative  courses  of 
action  cannot  be  identified,  the  situation  must  be 
classified  as  a  ‘worry’  and  not  an  actual  decision 
problem. 

2.  Identify  alternatives  -  determine  other 
alternatives.  The  identification  of  new  alternatives 
never  makes  a  decision  more  difficult  and  may  in 
fact  simplify  the  choice. 

3.  Establish  outcomes  -  "...  an  outcomes  is 
whatever  the  decision-maker  would  like  to  know  in 
retrospect  to  determine  how  the  problem  came  out. 

In  a  military  problem,  the  outcome  could  be  a 
complicated  list  of  casualties,  destruction,  and 
armament  expenditures"  (Matheson  and  Howard, 

1968:  27). 

4.  Select  system  variables  -  system  variables  are 
those  variables  upon  which  the  outcomes  of  the 
decision  depend.  System  variables  are  of  two 
types.  Decision  variables  are  system  variables  that 
are  under  the  decision-maker's  control.  State 
variables  are  the  system  variables  not  under  the 
decision-maker's  control. 

5.  Create  a  structural  model  -  specify  the 
relationships  of  the  system  variables  in  a  way  the 
captures  the  essential  interdependencies  of  the 
problem.  This  should  be  done  in  a  logically  precise 
manner  (e  g.  mathematics,  computer  simulation 
language) 


6.  Create  a  value  model  -  assigning  a  numerical 
(cardinal)  value  to  each  outcome.  If  the  resources 
to  be  allocated  by  the  decision  are  monetary,  the 
value  assigned  to  the  outcome  should  be  monetary 
also. 

7.  Create  a  time  preference  model  -  describing  the 
preference  of  the  decision-maker  for  the  time  value 
of  the  decision  action.  "Time  preference  is  the  term 
used  to  describe  the  human  phenomenon  of 
impatience. . . .  This  impatience  is  reflected  in  a 
willingness  to  consume  less  now  rather  than 
postpone  the  consumption"  (Matheson  and  Howard, 

1968:  29).  The  stream  of  value  reduce  to  account 
for  time  preference  is  called  ‘worth’. 

"The  modeling  part  of  the  deterministic  phase  thus  progresses 
from  the  original  statement  of  the  decision  problem  [the  decision 
basis]  to  a  formal  description  suitable  for  detailed  examination  by 
logical  and  computational  analysis  (Matheson  and  Howard,  1968:  29) 
Two  analyses  are  accomplished  as  part  of  the  deterministic  phase. 

1.  Decision  variable  sensitivity  analysis  -  By 
evaluating  the  range  of  value  each  decision  variable 
might  assume  (all  others  held  constant  at  a 
nominal  value),  determine  that  variable's  effect  on 
the  worth  of  the  decision.  Those  decision  variables 
which  have  little  effect  upon  the  decision  are  called 
'impotent'  and  can  be  removed  from  the  decision 
basis.  Other  decision  variables  remain  in  the  basis. 

2.  State  variable  sensitivity  analysis  -  As  above, 
each  state  variable  is  evaluated  over  the  range  of 
values  it  could  assume,  all  others  held  constant.  If 
the  state  variable  has  a  great  effect  on  the  worth 
of  the  decision,  then  must  be  further  analyzed. 

These  state  variables  are  called  'aleatory'  (i.e. 
dependent  upon  chance).  If  the  worth  of  the 
decision  changes  very  little,  then  the  variable  can 


be  fixed  at  its  nominal  value.  These  state  variables 
are  called  ’fixated'. 

In  summary,  during  the  deterministic  phase  the  variables 
relevant  to  the  problem/decision  are  defined  and  their  inter¬ 
relationships  determined.  Values  are  assigned  to  the  variables  and  a 
sensitivity  analysis  is  performed  to  measure  the  importance  to  the 
worth  of  the  decision  to  each  variable.  Uncertainty  is  not 
considered. 

Step  3:  Probabalistic  evaluation.  The  result  of  the  sensitivity 
analyses  accomplished  during  the  deterministic  phase  is  to  identify 
the  aleatory  state  variables.  "The  probabalistic  phase  determines  the 
uncertainty  in  value  and  worth  due  to  the  aleatory  variables” 
(Matheson  and  Howard,  1968:  30).  As  in  the  deterministic  phase,  the 
probabalistic  phase  is  divided  into  modeling  and  analyses  activities 
The  modeling  area  consists  of  the  following. 

1.  Encode  uncertainty  -  assign  probability 
distributions  to  each  of  the  aleatory  variables. 

"Either  the  decision-maker  or  someone  he 
designates  must  assign  the  probability  that  each 
aleatory  variable  will  exceed  any  given  value” 

(Matheson  and  Howard,  1968:  30).  For  detailed 
information  concerning  the  specific  techniques  used 
to  accomplish  this  see  Spetzler  and  Stael  von 
Holstein's  "Probability  Encoding  in  Decision 
Analysis",  1972:  601-625. 

2.  Encode  risk  preference  -  determine  the  decision¬ 
maker's  attitude  toward  the  worth  of  outcomes 
and  the  relationship  towards  uncertainty.  Risk  is 
the  concept  that  relates  the  worth  of  an  outcome 
to  the  uncertainty  associated  with  that  outcome. 

"A  risk-averse  decision-maker  is  willing  to  forego 
some  expected  value  in  order  to  be  protected  from 


the  possibilities  of  poor  outcomes"  (Matheson  and 
Howard,  1968:  43).  The  concept  of  risk  helps  to 
evaluate  choices  where  the  values  of  the  outcomes 
are  the  same  but  where  different  probabilities  for 
the  outcomes  have  been  determined.  For  a  detailed 
discussion  of  risk,  see  Howard's  "Risk  Preference", 

1970:  627-663. 

The  analysis  part  of  the  probabalistic  phase  consists  of  the 
following: 

1.  Develop  worth  lotteries  and  Certainty  Equivalent 
-  a  worth  lottery  is  the  probability  distribution  of 
worth  for  any  setting  of  the  decision  variables.  The 
certain  equivalent  worth  of  a  lottery  is  "...  the 
amount  of  worth  received  for  certain,  so  that  the 
decision-maker  would  be  indifferent  between 
receiving  this  worth  and  participating  in  the 
lottery"  or,  in  other  words,  taking  the  chance 
(Matheson  and  Howard,  1968:  32). 

2.  Determine  if  stochastic  dominance  exists  - 
determine  if,  for  any  value  of  a  aleatory  variable 
X,  there  is  a  greater  probability  of  one  alternative 
producing  a  worth  in  excess  of  that  worth  than 
will  another  alternative.  If  such  a  case  exists,  then 
the  first  alternative  is  said  to  stochastically 
dominate'  the  other.  If  such  is  the  case,  then  there 
is  no  need  to  determine  the  risk  preference  of  the 
decision-maker  with  respect  to  that  choice. 

3.  Measure  risk  sensitivity  -  determine  how  the 
certain  equivalent  worth  of  the  most  favorable 
alternatives  depend  upon  the  decision-maker's 
attitudes  toward  risk. 

In  summary,  during  the  probabalistic  phase,  probability 
assessments  are  made  for  each  variable  determined  to  be 
deterministically  sensitive  in  Step  2.  The  decision  maker's  attitudes 


toward  risk  are  assessed.  The  probabilities  and  risk  preferences  are 
added  to  the  model  and  sensitivity  is  measured  again. 

Step  4:  Basis  appraisal.  An  evaluation  is  made  of  the  three 

elements  of  the  basis  to  determine  if  additional  information 
gathering  or  model  review  is  required  before  the  decision  can  be 
made.  To  a  large  degree  this  involves  the  manager's  confidence  in 
the  information  about  the  decision  that  has  been  gathered  to  date, 
and  the  estimated  value  of  additional  information  that  may  be 
desired.  The  value  of  additional  information  is  a  marginal  analysis 
of  the  cost  versus  benefits  of  the  information  being  considered.  The 
object  is  to  determine  the  value  of  eliminating  uncertainty  in  the 
aleatory  variables. 

When  the  benefits  are  determined  to  exceed  the  cost  of  the 
information,  the  cycle  is  iterated  and  the  decision  basis  is  updated. 
This  process  of  iteration,  update,  and  appraisal  is  continued  until 
"...  the  value  of  new  analysis  and  information-gathering  will  be  less 
than  its  cost,  and  the  decision  to  act  will  then  be  made  (Matheson 
and  Howard,  1968:  26). 

Step  5:  Decision  and/or  iteration,  if  additional  information  is 
not  determined  to  be  required,  then  the  decision  can  be  made. 
Otherwise,  the  cycle  returns  to  Step  1. 

Before  proceeding  to  apply  the  decision  analysis  model  to  the 
situation  faced  by  the  acquisition  Program  Manager,  working  in  a 
large  bureaucratic  organization  such  as  the  Air  Force,  the 
assumptions  and  underlying  theoretical  foundations  of  decision 
analysis  will  be  reviewed.  Howard  has  stated  that  decision  analysis 


is  concerned  only  with  individual  decisions.  Group  decisions,  as  found 
in  a  large  organizations  like  the  Air  Force,  are  not  considered 
(Howard,  1983:  181). 

Raiffa,  on  the  other  hand,  addresses  this  group  decision  making 
situation  using  decision  analysis  in  some  detail.  His  discussion  centers 
on  the  applicability  of  individual  decision  making  behavioral 
assumptions  to  the  group,  or  organizational,  situation  (Raiffa,  1968: 
233). 

Summary 

This  chapter  has  laid  a  foundation  for  further  study  of  the 
decision  making  situation  faced  by  the  acquisition  Program  Manager 
The  organizational  structure  of  the  overall  acquisition  system  was 
shown  to  be  comprised  of  four  interrelated  sub-processes  These  sub¬ 
processes  were  then  shown  to  be  susceptible  to  the  influence  of 
environmental  forces  These  in  turn  complicated  the  decision  maxing 
task  of  the  acquisition  Program  Manager 

Management  theory  was  reviewed  in  terms  of  organizational 
structure  and  organizational  effectiveness  The  Air  Force  acquisition 
system  was  determined  to  be  essentially  bureaucratic  but  the 
consideration  of  the  open  systems  theory  and  the  socio-technical 
systems  theory  provided  additional  knowledge  about  the  complexities 
of  the  acquisition  situation 

Finally,  decision  analysis  was  offered  as  a  way  to  assist  the  PM 
is  the  decision  making  task  The  five-step  decision  analysis  cycle  was 
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briefly  explained.  However,  the  ready  applicability  of  the  decision 
analysis  cycle  to  large  organizations  was  left  in  question. 

The  following  chapter  will  present  a  method  for  analyzing  a 
recent  Air  Force  acquisition  decision.  The  decision  will  be  compared 
against  a  normative  organizational  decision  making  model  which  is 
developed  in  Chapter  IV.  Chapter  V  will  present  the  results  of  the 
analysis  and  Chapter  VI  will  summarize,  draw  conclusions  and 
present  recommendations. 


IH  Methodology 


Introduction 

This  research  effort  was  structured  to  determine  whether  the 
selective  application  of  decision  analysis  principles  and  procedures, 
within  a  coherent  normative  decision  making  framework,  could  be 
used  to  improve  the  Air  Force  acquisition  Program  Manager's 
decision  making.  Part  of  the  problem  was  to  determine  the  extent 
to  which  decision  analysis  tasks  were  already  being  accomplished. 

Specifically,  the  study  developed  a  normative  model  as  an 
overall  management  decision  making  framework.  The  normative 
model  was  based  upon  the  results  of  the  literature  review 
summarized  in  chapter  II  and  was  intended  to  be  suitable  for  use  by 
Air  Force  acquisition  Program  Managers.  Since  an  objective  of  this 
study  was  to  draw  conclusions  about  the  applicability  of  the  model 
as  a  guide  for  acquisition  PMs,  it  was  necessary  to  test,  or  validate, 
the  model  using  an  actual  Air  Force  acquisition  decision 

Therefore,  the  purpose  of  this  chapter  is  to  define  how  a  recent 
Air  Force  acquisition  decision  was  compared  to  the  normative  model 
so  that  the  applicability  of  the  model  could  be  determined  as  well  as 
the  extent  to  which  relevant  tasks  were  already  being  accomplished 
The  tasks  of  the  analysis  will  be  discussed,  variables  defined,  and 
statistical  tests  explained. 


Analysis  Tasks 


After  the  literature  review  was  completed,  it  was  determined 
that  an  important  part  of  this  study  would  be  the  choice  of  a  recent 
Air  Force  decision  for  use  as  a  descriptive  acquisition  decision  making 
baseline.  A  case  study  of  a  real  decision  was  needed  to  provide  the 
necessary  links  to  future  implementation  and  research. 

The  first  objective  of  the  case  study  was  to  provide  a  solid  and 
realistic  foundation  as  the  practical  point  of  reference  for  drawing 
conclusions  about  the  applicability  of  the  overall  normative 
framework  which  included  the  decision  analysis  principles.  Secondly, 
it  was  desirable  to  determine  if  existing  procedures  within  the 

! 

'  acquisition  system  provided  the  information  required  by  decision 

analysis.  Implementation  planning  for  a  new  system  must  consider 
the  total  situation  into  which  it  will  be  placed  in  order  to  have  any 
probability  of  success.  Implementation  planning  should  therefore 
begin  early  in  system  design  (Conner,  1985:  27). 

The  Air  Defense  Fighter  (ADF)  Competition  was  chosen  for 
several  reasons.  The  ADF  Competition  was  a  large,  complex 
program,  both  technically  and  programmatically.  The  decision 
resulted  from  the  efforts  of  many  people  from  numerous 
organizations  and  managerial  levels  within  those  organizations. 
Significant  issues  surrounded  the  competition,  several  of  which  were 
political.  These  issues  sprang  directly  from  the  acquisition  reform 
movement,  of  which  the  environmental  changes  discussed  in  the 
literature  review  were  prime  examples. 

i 
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The  ADF  Competition,  however,  was  initiated  by  the  unsolicited 
Northrup  F-20  proposal,  which  was  seen  by  many,  including  the 
Congress,  as  an  opportunity  to  increase  competition  in  tactical  fighter 
procurement  (Gordon,  1985:  1611).  This  aspect  could  cause  the 
competition  to  be  viewed  as  atypical  of  DOD  weapon  system 
acquisition  programs.  It  was  assumed,  however,  that  most 
acquisition  programs  are  affected  in  their  early  stages  by  defense 
industry  marketing  activity.  It  was  determined  that  it  would  be 
beyond  the  scope  of  this  study  to  attempt  to  ascertain  the  relative 
influence  upon  the  acquisition  system  of  these  various  influences. 
Additionally,  it  was  assumed  that  any  influence  there  was  would 
not  change  the  conclusions  of  the  study.  Instead,  it  is  simply 
another  example  of  the  complex  and  dynamic  nature  of  the 
management  of  weapon  system  acquisitions.  For  these  reasons  the 
ADF  program  was  judged  by  the  author  to  be  an  acceptable  example 
of  current  major  Air  Force  acquisition  decisions. 

The  ADF  decision  was  analyzed  from  approximately  the  point 
in  time  when  the  initial  unsolicited  bid  from  Northrup  Corporation 
was  received  by  the  Air  Force  to  just  prior  to  the  final  award  of  a 
contract  to  General  Dynamics  Corporation.  This  history'  was 
prepared,  and  organized,  according  to  the  sequence  of  steps  associated 
with  the  scientific  method  or  traditional  problem  solving  process: 
identify  the  problem,  define  alternative  solutions,  gather  data 
necessary  to  evaluate  the  alternatives,  evaluate  the  alternatives, 
and  finally  make  the  choice  (Borg  and  Gall,  1971:  7-8).  These  steps 
were  chosen  as  a  common  denominator  for  the  overall  analysis 


because  of  their  relative  simplicity,  comprehensive  nature,  and  the 
familiarity  managers  have  with  the  steps. 

Key  Variables.  Within  the  context  of  the  traditional  problem 

solving  phases,  an  analysis  was  accomplished  of  the  ADF  competition 
from  the  perspective  of  the  acquisition  Program  Manager  in  terms  of 
Air  Force  acquisition  decision  making.  The  analysis  focused  on  two 
key  variables:  the  creation  of  information  and  the  communication  of 
information.  These  key  variables,  information  and  communication, 
are  discussed  below. 

Information  Processes.  For  an  individual,  the  following 
table  defines  information  related  phases  of  a  decision  process  (Ebert  8, 
Mitchell,  1975:  67). 


Table  1 

Information  Use  During  a  Decision  Process 


Acquisition 

Processing 

Output 

Search  behavior 

Information 

Judgment 

utilization 

Within  an  organizational  context,  and  the  Air  Force  weapon 
system  acquisition  process,  the  information  processes  used  in  the 
problem  solving  process  become  critical.  The  purpose  of  this  decision 
making  variable  is  to  reveal  the  manner  in  which  ADF  Competition 
Program  Manager  created  and  processed  information.  The  study 
was  interested  in  determining  the  nature  of  information  search 
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activities  and  how  that  information  was  used  in  the  decision  making 
process.  The  judgment  phase  was  determined  to  be  not  applicable  to 
the  objectives  of  this  study. 

Communication  Processes.  "The  collection,  evaluation,  and 
transmission  of  information  can  vitally  affect  organizational  and 
individual  performance"  (Hellriegel  and  Slocum,  1974:  266).  The 
communication  process  determines  the  hierarchical  direction  of  the 
flow  of  information,  who  sends  and  receives  the  information,  the 
quantity  of  information  transmitted,  and  the  timing  of  the 
transmission.  Each  of  these  aspects  of  the  communication  process 
has  a  great  affect  upon  the  effectiveness  of  the  decision  making 
process.  In  the  Air  Force  acquisition  decision  making  process,  these 
communication  flows  are  often  very  formalized.  Informal 
transmission  mechanisms,  however,  cannot  be  ruled  out  a  priori  as 
not  having  influence  over  results.  The  analysis  sought  to  identify 
the  flows  of  communications  and  whom  the  communications  were 
with. 

Mitigating  Variables  Throughout  the  ADF  competition  problem 
solving  process,  and  associated  with  each  key  variable,  there  existed 
aspects  of  the  Air  Force  decision  making  context  that  were  identified 
and  discussed  with  regards  to  their  impact  upon  the  analysis. 
Inclusion  of  these  aspects  is  necessary  to  facilitate  an  understanding 
of  the  process  of  decision  making  as  a  group  process  within  an 
organization  These  aspects  are  termed  mitigating  variables  and  are 
discussed  below 


Organizational  Structure.  The  structure  of  the  Air  Force 
acquisition  process  refers  to  the  organizational  elements  assigned  the 
overall  responsibility  for  the  process,  the  definition  of  the  tasks  they 
are  to  accomplish,  and  the  formal  relationships  between  elements. 
This  structure  may  or  may  not  mirror  the  structure  of  information 
flows  defined  by  the  communication  process  although  there  is 
assumed  to  be  considerable  overlap.  The  key  issues  were  the  extent 
to  which  the  formal  organizational  structure  supported  the 
objectives  of  the  decision  making  process  and  whether  any  apparent 
structural  dissonance  significantly  impeded  the  efforts  of  the 
organization  to  solve  the  problem. 

Established  Procedures.  As  mentioned  above,  the  Air 

Force  major  weapon  system  acquisition  process  is  highly  formalized 
It  was  shown  in  the  iiterature  review  that  this  formalization  takes 
the  form  of  four  separate  but  inter-dependant  acquisition  sub¬ 
processes.  Many  governmental  agencies  have  enacted  regulations 
governing  how  these  sub-processes  are  to  proceed.  Given  the 
complexity  of  this  regulatory  environment,  these  procedures  may 
vary  from  one  problem  solving  phase  to  another,  and  may  have 
effected  one  key  decision  variable  differently  from  the  other. 

External  Influences  This  mitigating  variable  was 

primarily  concerned  with  political  influences  upon  the  decision 
making  process.  In  the  case  of  the  ADF  Competition,  these  influences 
originated  from  the  Congress,  the  media  ana  other  organizational 
elements.  External  influences  tended  to  be  transmitted  primarily 
through  the  budget  process. 


Personnel.  The  particular  personnel  assignments  in  affect 
at  a  given  time  can  greatly  influence  many  of  the  administrative 
decisions  made  during  a  problem  solving  process.  For  example,  a 
manager  with  a  substantial  engineering  background  might  react 
differently  to  a  certain  aspect  of  the  problem  than  would  a  pilot 
with  substantial  operating  experience.  Established  procedures  may 
or  may  not  have  always  corrected  for  this  tendency. 

Data  Categorization  Scheme.  The  data  from  the  analysis, 

throughout  the  traditional  problem  solving  steps,  was  categorized  in 
terms  of  the  key  variables  and  mitigating  variables.  The  scheme  to 
accomplished  this  task  was  content  analysis.  The  categorized  data 
were  then  correlated  to  the  primary  attributes  of  the  proposed 
normative  framework  (developed  in  the  next  chapter).  Since  this 
study  was  primarily  concerned  with  information  and 
communication,  and  since  it  was  presumed  that  significant 
communication  was  primarily  accomplished  via  the  transmission  of 
written  documents  from  one  organization  to  another,  the  source 
material  used  for  the  analysis  was  limited  to  the  documents 
maintained  by  the  ADF  Program  Manager  and  the  ADF  Competition 
management  team.  The  actual  Request  for  Proposal  (RFP)  issued  to 
the  interested  and  qualified  contractors  was  also  an  important 
documented  source 

"Content  analysis  is  a  method  of  studying  and  analyzing 
communications  in  a  systematic,  objective  and  quantitative  manner 
for  the  purpose  of  measuring  variables"  (Kerlinger,  1964:  544).  The 
first  step  in  doing  the  content  analysis  study,  was  to  define  the 


universe  of  content  to  be  analyzed.  For  the  ADF  Competition,  the 
universe  was  defined  as  the  collection  of  documents  assembled  by  the 
Program  Manager. of  the  competition.  This  documentation  included 
most  unclassified  correspondence  and  other  written  materials  from  2 
April  1985  until  31  October  1986.  The  RFP  was  also  defined  to  be 
within  the  universe. 

The  second  step  was  to  determine  the  appropriate  unit  for  the 
analysis  of  the  universe.  In  the  ADF  Competition,  the  units  were 
defined  as  the  individual  documents  themselves.  For  example,  a 
letter  from  the  Program  Manager  to  the  headquarters  would  be  a 
unit.  However,  when  a  document  had  attachments  that  were  by 
themselves  formal  communications  concerning  the  competition,  those 
documents  were  treated  as  separate  units.  For  example,  a  letter 
sent  to  the  Program  Manager  from  the  headquarters  would  be  a 
unit  as  would  an  attached  memorandum  from  the  DOD  to  the 
Secretary  of  the  Air  Force. 

The  next  step  was  to  break  down  the  universe  into  major 
categories.  Categories  are  the  first  level  of  the  actual  content 
analysis.  The  categories  chosen  for  the  ADF  Competition  were  the: 

1.  Control  category, 

2.  Problem  solving  steps, 

3.  Weapon  System  Acquisition  (WSA)  sub-processes, 

A.  Information  processes, 

5.  Communication  processes,  and 

6.  Critical  decision  analysis  tasks. 


Each  major  category  was  then  divided  into  subcategories.  All 
subcategories  are  shown  in  Figure  11.  Definitions  of  each  subcategory 
follow  below. 

Control  Category.  The  subcategories  under  the  category 
Control  follow. 

Book:  The  binder  the  document  was  located  in. 

Tab:  The  tab  the  document  was  filed  behind. 

Date:  The  date  of  the  document. 

To:  The  organization  to  which  the  document  was  primarily 
addressed. 

From:  The  organization  primarily  responsible  for  generating 
the  document. 

Subject:  The  stated  subject  or  title  of  the  document. 

Purpose:  A  brief  synopsis  of  the  intended  purpose  of  the 
document. 

Problem  Solving  Step  Category.  The  subcategories  under 
the  category  Problem  Solving  Step  follow 

Problem  Definition:  Determine  whether  the  document 
supported  the  activity  of  problem  definition 

Identification  of  Alternatives:  Determine  whether  the 
document  supported  the  efforts  of  the  decision  making  unit  and  the 
Program  Manager  to  identify  alternative  courses  of  action  Decisions 
made  during  the  definition  of  the  problem  phase  impact  how  the 
identification  of  alternatives  phase  will  progress.  Alternatives  m  the 
acquisition  process  take  on  two  forms.  First,  will  the  actual  program 
be  managed  as  a  sole  source  procurement  from  a  single  prime 


iiniiiorco*  AOf:  PnoQna™  Manager's  ,.nitc.  Program  Management 
Correspondence  File  un,ls*  Documents 


Categories 

Sub-categories 
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Control 

•Date 

Apr  85  thru  Oct  86  ( 19) 
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•From'  ^Subject 
•Purpose 

As  appropriate 

Problem  Solving 
Steps 

•Problem  •Identification 

Definition  of  Alternatives 

•Gather  •Evaluation 

Data  of  Alternatives 

•Decision 

YesorNo(2) 

Weapon  System 
Acquisition 

Sub -Processes 

•Requirements  *Budget 
•Acquisition  Management 
•Contracting 

Yes  or  No(  2 ) 

Information 

Processes 

•Environmental  •Provide 
Influence  Information 

•External  •information 

Policy  Source 

•External 

Tasking 

! 

Yes  or  No(  2 ) 

r 

•Direction 
of  Flow 
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Unknown  lUNk'  Outgoing 
•Internal  15.' 

Communication 
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Coordination 
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Coordination 

1 

1 
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Critical  Decision 
Analysis  Tasks 

•Outcomes 

•Preferences 

•Variables 

•Models 

•Uncertainty 
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contractor  or  as  a  competition  between  two  or  more  contractors. 
Second,  will  be  the  set  of  technical  approaches  for  the  conduct  of  the 
program  proposed  by  the  single  contractor  or  by  competing 
contractors. 

In  each  of  these  instances,  information  was  used  to  justify  the 
definition  of  alternatives.  Similarly,  the  existing  Air  Force  acquisition 
system  will  affect  this  process  through  its  specific  communication 
channels.  Documents  from  the  ADF  Competition  were  analyzed  to 
trace  the  flow  of  this  process. 

Gather  Data:  Determine  whether  the  document  supported  the 
data  gathering  activities  of  the  decision  making  unit.  Documents 
were  categorized  as  data  gathering  whether  the  data  were  gathered 
to  support  problem  definition,  alternative  identification,  or  for  some 
other  purpose. 

The  evaluation  of  the  alternatives  depended  heavily  on  the 
data  that  were  gathered  to  support  the  evaluation.  The  data  chosen 
depended  upon  the  sources  of  information  known  to  exist  by  the 
personnel  involved  in  the  decision  making  process  and  by  the 
mechanisms  employed  by  the  process  itself  to  find  and/or  generate 
data.  Similarly,  the  extent  to  which  the  existing  communication 
channels  allowed,  or  even  encouraged,  this  search  could  have 
significant  affects  on  the  eventual  conduct  of  the  evaluation  of 
alternatives. 

Evaluation  of  Alternatives:  Determine  whether  the  document 
supported  alternative  evaluation  activity.  The  purpose  of  the 
evaluation  phase  is  to  process  relevant  data  about  the  alternatives 
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into  useful  information  that  will  allow  an  informed  choice  to  be 
made.  The  success  of  this  processing  depends  upon  the  tools  available 
and  the  clarity  of  the  objectives  of  the  evaluation.  The  tools 
available  for  evaluations  vary  considerably  in  their  nature, 
assumptions  and  products.  The  use  of  the  tools  must  be  guided  by 
the  overall  objectives  of  the  evaluation  phase.  The  product  of  the 
tools  must  be  coherent  between  the  tools  and  between  the 
alternatives  evaluated.  The  ability  to  choose  between  alternatives  is 
contingent  upon  clear  comparisons  of  meaningful  information  about 
the  alternatives. 

Decision:  Determine  whether  the  document  established  a  final 
choice.  The  final  decision  is  based  upon  the  results  of  the  previous 
phases  of  the  problem  solving  process.  Weakness  in  any  of  the 
preceding  actions  will  undermine  the  quality  of  the  final  choice.  The 
choice  in  the  Air  Force  acquisition  process  involves  not  only  choosing 
an  alternative,  but  also  justifying  the  choice  to  others.  This  process 
of  justification,  and  its  anticipation  by  decision  making  participants, 
affects  the  actions  of  the  participants  throughout  the  process. 

Weapon  System  Acquisition  Process  -  Sub-Process 
Category.  The  subcategories  under  the  acquisition  sub-processes  are 
discussed  below.  The  sub-processes,  discussed  thoroughly  in  the 
literature  review,  included: 

Requirements  Sub-Process:  Determine  whether  the  document 
supported  the  preparation  of  a  validated  Statement  of  Operational 
Need  (SON). 
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Budget  Sub-Process:  Determine  whether  the  document 
supported  the  preparation  or  update  of  the  Program  Objective 
Memorandum  (POM). 

Acquisition  Management  Sub-Process:  Determine  whether  the 
document  supported  the  preparation  of  the  Program  Management 
Directive  (PMD)  or  whether  the  document  supported  the  activities  of 
the  Program  Manager  with  regards  to  the  general  management  of 
the  program  (i.e.  activities  relating  to  program  documentation  not 
covered  in  the  other  three  sub-processes). 

Contracting  Sub-Process:  Determine  whether  the  document 
supported  the  issuance  of  the  Request  for  Proposal  (RFP),.the 
subsequent  proposal  evaluation  and  the  contract  award  activities. 

Information  Process  Category.  The  subcategories  under 

the  information  process  category  are  defined  below. 

Environmental  Influence:  Determine  whether  the  document 
constituted  an  influence  upon  the  conduct  of  the  program  originating 
in  the  environment. 

Provide  Information:  Determine  if  the  document  was  intended 
to  convey  information  to  others, 

Information  Source:  Determine  whether  the  document  was 
intended  to  be  used  as  a  reference  source  for  information. 

External  Tasking:  Determine  if  the  document  constituted  a 
tasking  of  the  specific  program  (ADF  Competition)  by  an  organization 
external  to  the  immediate  decision  making  unit  (e.g.  DOD,  Congress, 
etc.). 
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External  Policy:  Determine  if  the  document  reflected  general 
policy  guidelines  applicable  to  several  programs. 

Information  Search  Activity:  Determine  if  the  document 
initiated  a  search  for  information  (i.e.  did  the  document  request 
information  from  another  organization  to  support  the  program). 

Communication  Process  Category.  The  subcategories 

under  the  communication  process  category  are  defined  below. 

Direction  of  Flow:  Determine  if  the  document  was  incoming  or 
outgoing  with  respect  to  the  program  organization.  Determine  if  the 
document  was  used  as  an  internal  medium.  If  the  document  was 
none  of  these,  list  as  Not  Applicable  (NA).  If  the  document  flow  could 
not  be  determined,  list  as  Unknown  (UNK). 

Intra-organizational  Coordination:  Determine  whether  the 
document  was  used  to  gain  internal  program  consensus  on  an  issue. 

Extra-organizational  Coordination:  Determine  whether  the 
document  was  used  to  gain  consensus  from  relevant  external 
organizations  with  respect  to  an  issue. 

Critical  Decision  Analysis  Tasks  Category.  Listed  below 

are  the  subcategories  under  the  critical  decision  analysis  tasks 
category. 

Outcomes:  Determine  if  the  document  provided  information 
that  could  support  the  identification  of  possible  decision  outcomes. 

Preferences:  Determine  if  the  document  provided  information 
that  could  support  the  definition  of  organizational  preferences. 

Variables:  Determine  if  the  document  provided  information 
concerning  decision  and/or  system  variables. 


Models:  Determine  if  the  document  provided  information 
concerning  the  evaluation  of  alternatives  using  quantitative  analysis 
(e.g.  computer  simulations) 

Uncertainty:  Determine  if  the  document  acknowledged  that 
future  events  were  uncertain. 

Probabilities:  Determine  if  the  document  provided  information 
about  the  probability  of  a  future  event  occurring. 

Values:  Determine  if  the  document  provided  information  about 
the  value  placed  upon  a  decision  outcome  by  the  organization. 

Risk:  Determine  if  the  document  provided  information  that 
acknowledged  the  position  of  the  organization  on  the  possible 
negative  results  that  could  follow  the  decision. 

Depending  on  the  subcategory,  the  method  to  quantify  the 
categorization  varied.  For  the  subcategory  "date",  there  were 
nineteen  elements  ranging  from  April  1985  to  October  1986.  The 
remaining  control  subcategories  contained  information  relevant  to 
locating  and  describing  the  document  but  were  not  used  in  the 
statistical  analysis.  The  direction  of  flow  subcategory  had  five 
elements:  not  applicable,  incoming,  unknown,  outgoing,  or  internal 
The  remaining  subcategories  were  defined  by  either  a  yes  or  no 
answer. 

Data  Analysis  Approach.  Insofar  as  the  data  obtained  from  the 
content  analysis  described  above  was  nominal  data,  it  was  necessary 
to  define  a  statistical  test  of  correlation  that  was  appropriate.  Siegel 
has  stated  that  "...  the  most  common  measure  of  association  for 
nominal  data  is  the  contingency  coefficient,  C,  a  nonparametric 
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statistic"  (Siegel,  1956;  23).  The  contingency  coefficient  measures  the 
degree  of  association  between  two  variables  (Siegel,  1956;  195-202; 
Harnett,  1982;  718;  and  McClave  and  Benson,  1985;  799).  The 
relevant  assumptions  were  that  all  the  data  were  nominal,  that  no 
fewer  than  20$  of  the  contingency  table  cells  have  an  expected 
frequency  of  less  than  five,  and  that  no  cell  had  an  expected 
frequency  of  less  than  one  (Siegel,  1956:  201). 

The  data  were  analyzed  in  the  Statview  512+  software  package 
using  the  contingency  table  function  while  varying  the  definition  of  x 
and  y  variables.  The  software  produced  contingency  tables  in  the 
following  forms;  observed  frequency  table,  percent  of  rows  table, 
percent  of  column  table,  and  an  expected  values  table.  Also  created 
was  a  table  listing  the  degrees  of  freedom,  the  contingency 
coefficient,  and  the  p-value  for  that  test  (Cuneo  and  Feldman,  1986 
137-140). 

The  tests  run  had  degrees  of  freedom  of  18,  4,  or  1.  The  null 
hypothesis  (  Ho  )was  that  the  variables  compared  were  statistically 
independent.  The  alternative  hypothesis  (  Ha )  was  that  the 
variables  compared  were  dependant.  The  rejection  region,  at  a  .05 
ievel  of  significance,  was  defined  at  the  following  values  of  chi  square 
(  X2):  28.9  for  18df,  9.49  for  4df,  and  3.84  for  ldf  (Beyer,  1968.  294). 

Tests  were  run  in  accordance  with  Figure  11.  Shown  are  the 
categories  (and  two  subcategories  -  time  and  flow)  which  defined  the 
comparisons.  These  comparisons  were  chosen  because  they  were 
relevant  to  the  objectives  of  determining  if  the  normative  model's 
tasks  could  be  accomplished  in  the  system  currently  in  existence. 
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Figure  11 

Matrix  of  Variable  Comparisons 
For  example,  the  analysis  would  show  whether  the  decision 
analysis  tasks  of  determining  values  correlated  with  the  acquisition 
requirements  sub-process.  Or,  it  might  show  that  requirements 
related  documentation  occurred  early  in  the  decision  making  process. 

Summary 

This  chapter  defined  the  method  of  analysis  of  the  chosen 
acquisition  decision  example  and  its  documentation.  The  overall 
analysis  relied  upon  content  analysis  to  characterize  and  categorize 
the  data  .  The  data  were  then  statistically  analyzed  using 
contingency  tables  and  the  contingency  coefficient.  The  intent  of  the 
comparisons  was  to  identify  possible  significant  correlation  between 
relevant  variables. 


IV  Model  Development 


Introduction 

In  this  chapter,  the  five-step  decision  analysis  cycle  introduced 
in  the  literature  review  is  decomposed  to  focus  on  those  facets  of  the 
methodology  critical  to  its  transition  from  an  individual  decision 
making  context  to  that  of  group  decision  making  in  an  organization. 
The  decision  analysis  cycle  is  then  recomposed  into  a  form  consistent 
with  organizational  decision  making  and,  lastly,  evaluated  for 
theoretical  applicability  to  Air  Force  acquisition  decision  making  prior 
to  the  comparative  analysis  of  the  ADF  competition  in  Chapter  V 

Decomposition 

Decision  analysis  principles  are  based  upon  theoretical 
assumptions  and  groundrules  for  their  application  Three  aspects  of 
decision  making,  as  they  pertain  to  the  use  of  decision  analysis  in  an 
organization,  are  discussed  below  Each  aspect  was  chosen  because  of 
its  significance  to  the  application  of  decision  analysis  principles  in  a 
group  decision  making  situation  as  typically  found  in  bureaucratic 
organizations. 

Decision  Definition.  As  mentioned  in  Chapter  II,  Howard's  view 
of  decision  making  excludes  explicit  consideration  of  a  group  process 
Howard  mentions  an  individual,  acting  alone,  as  the  decision  maker 
This  individual  is  able  to  provide  a  single  set  of  preferences, 
alternatives,  outcomes,  and  probabilities  that  the  decision  analyst  can 
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use.  Similarly,  there  exists  a  single  attribute  (usually  money)  that 
can  be  used  to  unambiguously  evaluate  and  rank  the  alternatives. 
Once  the  optimum  alternative  is  determined,  a  single  irrevocable  act 
implements  the  decision  (Howard,  1983:  181-184). 

In  a  group  decision  making  context,  the  decision  making  process 
must  often  balance  competing  goals.  These  goals  may  arise  from  the 
different  constituencies  being  served  by  the  organization  or  from  an 
inability  to  discriminate  between  multiple  goals  (Steiner,  et  al,  1982: 
517-522).  Consider  the  four  generic  acquisition  goals  of  cost,  schedule, 
performance,  and  reliability  &  maintainability  as  an  example  of  this 
goal  conflict.  There  may  exist  simultaneous  requirements  to  increase 
an  aircraft's  operational  range,  decrease  its  life  cycle  cost,  and 
stabilize  its  gross  takeoff  weight.  The  ability  to  arrive  at  a  single 
measure  of  the  effectiveness  of  alternatives  to  satisfy  these  goals, 
agreeable  to  different  individuals  and  organizational  elements 
(particularly  in  the  nonprofit  arena),  is  operationally  difficult 
(Anthony  &  Young,  1984:  42;  and  Harrison,  1974:  554). 

Organizational  decision  processes  also  include  reviews  of  the 
decision  once  it  has  been  made.  There  can  be  a  number  of  reviews 
before  the  actual  decision  is  validated  by  the  corporate  whole. 
Drucker  terms  this  activity  the  'selling’  of  a  decision  (Drucker,  1974: 
468).  In  the  Air  Force,  a  primary  vehicle  in  making  an  important 
decision  is  ’briefing’  that  decision  to  other  interested  organizations. 
The  concept  of  'selling'  is  also  discussed  by  Loving.  The  interface 
with  other  organizational  elements  is  said  to  be  critical  because  such 
"...  coordination  serves  to  insure  that  the  left  hand  knows  what 
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the  right  hand  is  doing  and  agrees  -  -  so  that  the  staff  then  acts  in 
concert"  (ACSC,  1986:  22)  [Emphasis  added].  As  an  example, 
important  resource  allocation  decisions  must  be  handled  in  a  formal 
and  specified  manner  using  the  procedures  dictated  by  the  Planning, 
Programming,  and  Budgeting  System  (PPBS)  and  the  Air  Force  Board 
Structure,  as  discussed  in  Chapter  II. 

Decision  analysis,  if  it  is  to  be  implemented  within  an 
organization,  must  explicitly  account  for  this  process  if  it  is  to  be 
successful.  The  definition  of  a  decision,  at  least  in  the  large 
bureaucratic  organization,  must  recognize  the  formal  coordination 
process  in  which  decisions  are  formulated,  resolved,  and 
implemented. 

Rationality.  The  concept  of  rationality  can  be  associated  with 
the  decision  itself  or  with  the  process  followed  to  arrive  at  the 
decision.  A  decision  itself  is  rational  if  it  maximizes  the  achievement 
of  a  goal  within  environmental  limitations  (Dahl  &  Lindblom,  1953: 
38).  Rational  decision  making  as  a  process,  however,  concerns  the 
procedures  that  should  be  followed  to  arrive  at  a  decision.  This 
normative  aspect  is  consistent  with  the  basic  principles  of  decision 
analysis  (Matheson  &  Howard,  1983:  25). 

To  make  rational  decisions,  the  following  are  required  (Hellriegel 
&  Slocum,  1974:  152): 

1.  The  search  for  and  assimilation  of  all 
information  relevant  to  the  decision  issue. 

2.  The  ability  to  determine  preferences  according  to 
some  measuring  device, 


3.  The  ability  to  select  the  alternative  which 
maximizes  the  decision  maker's  satisfaction. 

The  availability  of  information  of  the  right  kind  and  at  the 

right  time  to  support  the  decision  making  process  is  problematic. 

One  of  the  key  reasons  for  supporting  decision  analysis  in  the 

organization  is  to  increase  the  decision  maker's  ability  to  bring  to 

bear  upon  the  decision  the  best  information  in  the  optimum 

structure  available  (Raiffa,  1968:  268-272).  Remembering  that  the 

definition  of  information  discussed  in  Chapter  11  called  for  a 

structuring  of  data  at  a  point  in  time  by  a  specific  person  to  serve  a 

specific  purpose,  decision  analysis  must  then  provide  support  for  the 

accomplishment  of  this  activity  by  the  Program  Manager 

Additionally,  since  information  is  a  scarce  resource,  the  decision 

analysis  cycle  must  provide  and  utilize  this  information  efficiently 

The  cycle  must  not  be  allowed  to  iterate  beyond  the  point  when  the 

value  of  additional  information  exceeds  its  utility 

However,  whether  or  not  the  PM  is  successful  depends  not  only 

upon  the  inherent  rationality  of  the  process  but  also  upon  how  well 

the  process  is  implemented  within  the  organization  to  begin  with. 

To  incorporate  the  principles  of  decision  analysis  into  an  organization 

requires  that  management  data  to  support  the  process  be  made 

available  and  that  the  personnel  involved  with  the  cycle  understand 

not  only  its  strengths  but  also  its  weaknesses  (Raiffa,  1968:  262) 

Utility  theory  provides  a  measure  of  the  preferences  of  a 

decision  maker.  Techniques  exist  that  describe  how  to  document  the 

utility  function  of  a  decision  maker  through  a  question  and  answer 

session  The  questions  employ  simple  lotteries  (i.e.  sets  of  choices)  to 
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estimate  the  utility  function  -  the  amount  of  'goodness'  the  decision 
maker  feels  as  value  increases  (Howard,  1983:  601-625).  As 
mentioned  above,  Howard's  view  of  decision  analysis  excludes  group 
decision  making.  In  an  organization,  however,  this  may  often  be  the 
governing  situation  (Steiner,  et  al,  1974:  520). 

Raiffa  addresses  group  decision  making  by  contrasting  two 
divergent  views  of  optimization.  On  the  one  hand,  the  group  is  able 
to  arrive  at  'compromise'  probabilities  and  utilities  which  can  then 
be  used  just  as  if  they  came  from  an  individual  with  no  change  to 
the  decision  analysis  process.  On  the  other,  the  group  debates  and 
discusses  the  issues  underlying  the  decision  and  arrives  at  a 
'consensus'  decision  such  that  no  other  alternative  will  make  any 
one  individual  better  off  that  does  not  make  someone  worse  off. 

This  condition  is  known  as  Pareto-optimality  (Raiffa,  1968  234). 

The  degree  to  which  one  method  of  gaining  group  consensus 
about  organizational  preferences  is  better  than  another  will  depend 
upon  the  organization  into  which  the  methods  are  asked  to  work 
(Heilnegel  &  Slocum,  1974:  151-152).  In  the  Air  Force,  while  there  is  a 
strong  tradition  of  decision  making  by  group  consensus,  the 
specification  of  organizational  values  a  priori  is  weak.  While 
directives,  regulations,  office  instructions  and  Commanders'  policy 
letters  exist,  they  tend  to  be  either  general  in  nature  or  not 
applicable  in  all  cases. 

Another  rationality  issue  deals  with  the  rules  employed  to 
determine  which  of  the  decision  alternative  being  considered  is  the 
optimum.  Simon  maintains  that  the  theoretical  requirements  for 


long-run  equilibrium  (i.e  an  overall  optimum,  a  global  maximum) 
implied  by  maximizing  are  not  realistic.  He  states  that  "  there  is 
no  a  priori  reason  to  assume  long-run  equilibrium”  (Simon,  1959 
263)  given  the  complex  and  dynamic  environment  the  organization 
faces.  Instead,  ’satisficing'  decision  making  requires  only  a  limited 
search  of  alternatives  with  the  optimum  being  the  one  that  is  most 
acceptable  but  not  necessarily  the  best.  He  goes  on  to  say  that 
satisficing  models  of  decision  behavior  more  closely  relate  to  how 
individuals  in  firms  actually  make  decisions  (Simon,  1959  262-265). 

The  satisficing  concept  is  worth  considering  here  insofar  as 
invalid  assumptions  operative  during  organization  design,  or  used  to 
support  the  mandates  of  a  new  directives  and  management  systems, 
stand  to,  at  best,  confuse  those  charged  with  making  decisions  within 
that  structure.  Alternative  assumptions  might  alleviate  bottlenecks 
and  false  starts  from  occurring  iater  on  during  actual  decision 
making  activity.  The  PM  might  not  have  to  improvise  as  much  or 
spend  time  finding  ways  to  circumvent  an  unrealistic  and 
constraining  system. 

In  summary,  the  theoretical  concept  of  rationality  through 
maximizing  does  not  appear  ideally  suited  for  use  in  assessing  the 
decision  making  processes  of  large  organizations.  Rationality  based 
upon  rules  of  satisficing,  however,  is  a  descriptive  model  that  offers 
little  in  the  way  of  specific  guidance  for  a  Program  Manager  about 
to  initiate  a  decision  making  process.  Still,  strict  rationality  as  a 
governing  assumption,  implies  too  rigid  a  view  of  the  decision  making 
process  and  should  not  be  used  to  support  organizational  design 


decisions  (e.g.  structural  alignments  and  procedural  definitions)  This 
is  particularly  true  given  the  group  context  within  which  the 
decision  making  process  is  carried  out  and  the  significant 
environmental  influences  with  which  the  process  must  contend. 
Loosening  the  assumptions  of  rationality  allow  a  more  realistic  and 
flexible  process  to  be  implemented  within  the  organization  resulting 
in  more  efficient  decisions  concerning  the  value  of  additional 
information  generation  activities.  Raiffa  underscored  that  point 
when  he  wrote  that  "  .  .  .  a  precise  answer  to  the  wrong  question  is 
not  nearly  so  desirable  as  an  incomplete  answer  to  the  right  one 
(1968:  265). 

Politics.  When  assessing  problems  in  the  context  of  decision 
analysis  in  organizations,  there  are  two  dimensions  to  consider.  The 
first  is  the  technical  dimension  of  the  decision.  It  deals  with  the 
specific  facts  pertaining  to  the  system  or  systems  under  study.  The 
technical  dimension  is  the  area  that  receives  most  of  the  attention  of 
'management  scientists'. 

In  a  recent  talk  on  the  application  of  management  science  to 
the  military,  Professor  Gene  Woolsey  introduced  another  dimension. 
He  pointed  out  that  while  the  technical  dimension  is  important  in 
linking  the  problem  with  its  solution,  so  too  is  the  political  dimension. 
The  political  dimension  embodies  consideration  of  the  organization 
and  how  the  solution  decided  upon  must  eventually  be  implemented 
Woolsey  stated  that  the  traditional  emphasis  of  management  science 
is  on  the  technical,  while  the  political  is  where  the  real  emphasis 
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should  be.  Similar  to  Drucker,  Woolsey  believes  that  the  “...right 
answer  unused  is  the  wrong  answer"  (Woolsey,  1987). 

As  introduced  in  the  literature  review,  the  open  systems  and 
socio-technical  models  of  management  both  explicitly  address  the 
environment  within  which  the  organization  must  make  its  decision. 
In  the  open  systems  model,  the  system  itself  influences  and  is 
influenced  by  the  environment.  Inputs  are  received  from  the 
environment  and  the  system's  outputs  are  provided  to  the 
environment.  The  socio-technical  system  model  includes 
environment  as  a  discrete  input  along  with  human,  technical  and 
managerial  inputs  which  are  also,  at  least  indirectly,  received  from 
and  influenced  by  the  environment.  The  systemic  outputs  of 
production  and  innovation  appear  to  be  analogous  to  the  outputs  of 
the  systems  model.  The  socio-technical  model  adds  the  dimension 
more  clearly  of  the  environment  also  acting  upon  the  social  and 
technical  elements  of  socio-technical  behavior.  Interactions  and 
sentiments  are  obviously  areas  of  social  behavior  subject  to  the 
influences  of  environment.  Technology  is  also  subject  to  the 
influence  of  the  environment. 

As  discussed  in  Chapter  II,  the  four  organizational  sub-processes 
at  work  in  the  overall  weapon  system  acquisition  process  can  be 
affected  by  changes  in  the  environment.  Only  by  addressing 
throughout  the  decision  making  process  the  affects  upon  the  decision 
maker  of  the  organizational  politics  involved  in  coordinating 
information  and  obtaining  a  consensus  which  validates  the  decision, 
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wiil  the  process  be  able  to  efficiently  arrive  at  an  effective  solution 
to  the  specific  problem  under  review. 

The  discussion  above  emphasizes  the  following  points  that 
should  be  considered  when  implementing  decision  analysis  into  an 
organization.  First,  the  implementation  should  allow  for  group 
decision  making.  This  should  include  gaining  group  consensus  on  all 
aspects  of  the  decision  to  be  made.  The  group  should  agree  on  the 
definition  of  the  problem,  the  nature  of  the  decision,  the  alternative 
solutions,  what  data  are  applicable,  what  evaluation  methods  will  be 
used,  and  what  decision  will  be  made.  Second,  the  implementation 
must  be  structured  with  explicit  rules  that  acknowledge  the  cost  of 
additional  information.  Program  Managers  should  have  the  means 
to  make  rational  decisions  within  a  framework  that  processes 
information  requirements  rationally.  Third,  the  implementation 
should  address  the  open  systems  nature  of  organizational  decision 
making.  Numerous  constituencies  impact  the  alternatives  considered 
and  the  choices  made.  Provisions  for  easy  communication  of  decision 
relevant  information  between  and  among  all  decision  participants 
should  be  provided. 


Recomposition 

The  baseline  five-step  decision  analysis  cycle  offers  the  decision 
maker  increased  effectiveness  through  structured  informat,  m 
generation  and  transfer.  As  discussed  above,  however,  the  five-step 
cycle  cannot  be  used  directly  in  an  organizational  context,  such  as 


that  faced  by  the  acquisition  Program  Manager.  The  DOD  acquisition 
context  requires  group  decisions  in  a  complex  environment  that 
concern  problems  which  involve  the  differing  requirements  of  the 
four  acquisition  sub-processes  and  thus  cannot  always  be  optimized 
by  simple  maximization.  Therefore,  the  decision  analysis  cycle  must 
be  restated  in  a  form  that  accommodates  the  organizational  decision 
making  situation  of  the  Program  Manager. 

The  Organizational  Decision  Analysis  Model  (ODAM)  will  be 
discussed  in  a  way  that  parallels  the  general  systems  theory 
approach  to  viewing  organizations.  The  systems  approach  sees 
organizations  as  open-systems.  The  organization  is  made  up  of 
interdependent  parts  that  both  contribute  outputs  to,  and  receive 
inputs  from,  the  whole.  The  entire  organization  is  also 
interdependent  with  its  external  environment.  The  organization,  as 
an  open-system  (at  all  levels),  takes  inputs  of  information,  energy, 
and  materials  and  transforms  these  by  some  process  into  outputs  or 
products.  Like  the  inputs,  the  outputs  can  take  the  form  of 
information,  energy  and  materials  (Hellriegel  8,  Slocum,  1974:  65). 

While  the  ODAM  borrows  its  general  form  from  the  open 
systems  theory  of  management,  the  theory  of  bureaucracy  and  the 
socio- technical  systems  theory  make  valuable  contributions.  An 
important  caveat  of  this  study  was  that  radical  prescriptions  for 
organizational  change,  whether  in  terms  of  structure  or  procedures, 
would  not  constitute  a  realistic  solution  for  the  acquisition  PM.  It 
would  not  be  politically  feasible  to  attempt  to  implement  such 
changes.  As  Harrison  stated,  the  military  is  a  bureaucracy  with 
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long-standing  traditions  concerning  its  methods  of  management  and 
decision  making  (Harrison,  1978:  555,  598).  This  is  particularly  true 
about  how  the  military  determines  objectives,  values,  and 
preferences.  The  ODAM  was  built  based  upon  the  premise  that  little 
or  no  significant  change  must  be  required  to  use  the  decision  analysis 
techniques. 

Socio-technical  systems  theory  addresses  concerns  not  usually 
found  in  solutions  to  the  military's  organizational  decision  making 
problems.  Those  concerns  are  the  social  elements  of  personal 
interactions  and  sentiments,  and  the  technical  element  of  the 
processes  used  in  the  organization.  For  example: 

1.  Program  Managers  vary  from  individual  to 
individual. 

2.  The  staff  of  the  Program  Office  will  vary  from 
program  to  program. 

3.  The  problems  faced  and  the  resulting  decisions 
that  will  be  required  will  vary  from  time  to  time. 

Methods  for  structuring  data  and  improving  decision  making 
processes  should  explicitly  address  these  realities.  Similarly,  the 
individuals  participating  in  the  decision  making  process  will 
inevitably  introduce  their  own  particular  biases  and  inconsistencies 
into  the  process.  Therefore,  the  structured  process  proposed  should 
ensure  that  all  aspects  of  the  problem  are  explicitly  documented  so 
that  at  any  step  in  the  decision  making  process,  other  decision 
making  participants  within  the  group  can  challenge  assumptions, 
criteria,  or  evaluations  so  that  an  improved  solution  can  be  found. 
Finally,  as  discussed  in  Chapter  II,  the  four  acquisition  sub-processes 
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at  work  in  the  weapon  system  acquisition  arena  are  interwoven. 

The  interdependent  nature  of  the  sub-processes  should  be  taken  as 
given  and  all  aspects  of  their  interaction  explicitly  addressed  during 
the  decision  making  process. 

Another  approach  exists  to  define  rationality  with  respect  to 
decision  making  by  organizations.  Simon  states  that,  in  general, 

. .  rationality  is  concerned  with  the  selection  of  preferred 
behavior  alternatives  in  terms  of  some  system  of  values 
whereby  the  consequences  of  behavior  can  be  evaluated 
(Simon,  1976:  75). 

The  concept,  termed  ‘bounded  rationality',  goes  on  to  offer 
three  elements  to  explain  the  conflicts  and  differences  between 
individuals  making  decisions  (Hellriegel  8,  Slocum,  1974:  153-155). 

The  elements  are: 

1.  Instead  of  seeking  a  single  optimum  action,  an 
individual  might  establish  a  limited  range  of 
alternatives  that  would  result  in  satisfactory 
outcomes. 

2.  Individuals,  or  organizations,  are  assumed  to 
undertake  a  restricted  search  for  possible 
alternatives. 

3.  Factors  outside  the  control  of  the  decision 
maker(s)  will  affect  the  outcome  of  the  decision 

As  a  prescription  for  designing  an  organizational  decision 

making  framework,  the  concept  of  bounded  rationality  at  least 

partially  resolves  the  issue  that  decision  makers,  whether  individuals 

or  organizational  entities,  will  frequently  have  incomplete  knowledge 

on  the  decision  problem  they  face  Simon  concludes  that  the  theory 

of  the  organization  must  include  as  a  basic  element  the  fact  that  the 


function  of  communicating,  and  the  technology  within  the 
organization  for  its  accomplishment,  are  critical  considerations  for 
effective  organizational  decision  making  (Simon,  1976:  243-244). 

The  heart  of  the  debate  with  respect  to  rules  of  rationality 
mostly  concerns  the  limits  that  the  organization,  or  individual 
decision  maker,  is  willing  to  go  to  in  order  to  acquire  marginally 
beneficial  information.  If  the  organizational  norm  is  to  get  all  the 
information,  regardless  of  the  cost,  this  would  be  considered  a 
rational  process.  Too  often  organizations  are  encouraged  to  continue 
information  search  activity  because  of  the  fear  that  someone  may  be 
able  to  successfully  challenge  the  problem  formulation  and 
subsequent  decision.  Additional  information  search  is  intended  to 
minimize  the  possibility  of  that  occurring.  A  more  profitable  use  of 
the  resources  expended  thus  would  be  to  structure  the  decision 
process  rationally  from  the  beginning  so  that  subsequent  defense  and 
justification  of  the  decision  would  be  easier  (although  still  not  certain 
of  success) 

The  decision  analysis  cycle,  unaltered,  would  contribute 
significantly  to  the  organization's  efforts  to  communicate  during  its 
decision  making  process.  By  viewing  the  cycle,  somewhat  modified, 
as  a  entity  in  and  of  itself  fitting  into  a  larger  organizational  decision 
analysis  process,  the  limitations  discussed  above  can  be  minimized. 
The  ODAM  is  discussed  below  in  terms  of  inputs,  process,  and 
outputs. 

Three  Levels.  The  DOD  acquisition  organizational  structure  is  a 
large  hierarchical  bureaucracy.  The  boundaries  of  the  acquisition 


bureaucracy,  viewed  as  an  open  system,  can  be  defined  in  terms  of 
those  organizational  elements  that  fall  within  the  system  and  those 
that  are  part  of  the  systems  environment.  The  environment  is 
defined  as  those  elements  that  affect  the  system  but  are  not 
controllable  by  the  system.  For  this  study,  because  of  its  focus  on 
the  Air  Force  acquisition  PM,  the  system  was  defined  as  the  Air 
Force  acquisition  community.  Figure  12  below  illustrates  the 
breakout  of  elements  into  either  the  environment  or  the  system 


Figure  12 

System  Boundaries 


Within  the  total  Air  Force  acquisition  system,  are  many  nested 
subsystems.  The  Requirements,  Budget,  Acquisition  Management 
and  Contracting  sub-processes  are  examples  of  such  nesting.  The 
system  also  contains  levels  of  organization  directives  that  prescribe 
how  the  activities  of  each  of  the  sub-processes  are  to  occur  and  be 


documented.  While  at  the  overall  organizational  level  (directives  and 
regulations)  and  at  the  supporting  level  (operating  instructions) 
detailed  guidance  is  provided  to  the  PM,  a  framework,  is  not  provided 
to  the  decision  maker  covering  the  complex  interface  of  these  sub¬ 
processes  and  levels  of  guidance.  The  ODAM  is  structured  in  nested 
levels  in  a  complimentary  fashion  to  alleviate  this  problem. 
Specifically,  the  ODAM  includes  a  Managerial  Level  as  the  interface 
between  the  Organizational  Level's  goal  oriented  directives  and  the 
Supporting  Level's  detailed  instructions.  Figure  13  illustrates  the 
relative  positions  of  the  three  levels  and  the  communication  activity 
that  takes  place  between  each. 


Figure  13 
ODAM  Levels 

The  three  levels  proposed  are  an  Organizational  Level,  a 
Managerial  Level  and  a  Supporting  Level.  Each  level  is  defined 


below.  The  definitions  are  referenced  to  the  perspective  of  the  Air 
Force  acquisition  Program  Manager. 

Organizational  Level.  The  organizational  level  is 

analogous  to  the  broader  organizational  processes  and  sub-processes. 

It  is  at  this  level  that  program  resources,  goals,  objectives  and  many 
of  the  constraints  are  determined.  Figure  14  illustrates  the  make  up 
of  the  organizational  level. 


Figure  14 

Organizational  Level 


The  organizational  level  is  also  the  primary  location  of  the 
organization's  boundary  scanning  activity  (i.e  the  activity  to  gather 
and  input  into  the  organization  relevant  evironmental  changes) 
Coordination,  compromise  and  consensus  at  the  Organization  Level 
involve  organizational  elements  within  the  Air  Force  but  outside  the 
control  of  the  Program  Manager  Procedures  are  described  in 
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moderate  detail  bat  tend  to  be  relatively  inflexible.  Positional 
authority  is  very  important  while  expert  knowledge  is  less  critical 
(Harrison,  1978:  553).  Situations  at  the  Organizational  Level  are  very 
dynamic  with  the  decision  focus  changing  often.  The  decision  making 
environment  is  most  complex  at  this  level. 

Managerial  Level.  The  Managerial  Level  is  the  level  at 

which  the  Program  Manager  works.  Most  resources  employed  at 
this  level  are  either  under  the  PM's  direct  control  or  are  intended  to 
support  the  program  via  an  organizational  matrix  or  some  other  like 
device.  Figure  15  illustrates  the  Managerial  Level.  It  incorporates 
the  five-step  baseline  decision  analysis  cycle. 


Figure  15 
Managerial  Level 


The  decision  analysis  cycle  constitutes  the  primary  tool 
available  to  the  PM  to  organize  data  and  solve  the  problem.  The 
PM  will  take  from  the  macro-level  data  necessary  to  proceed  with 
the  decision  making  process.  Such  data  includes  directives  concerning 
organizational  preferences  (e.g.  Program  Management  Directives  and 
Statement  of  Operational  Need),  procedural  guidelines  concerning  how 
the  answer  must  be  formulated  (e.g.  POM  submission  instructions), 
coordination  required,  and  usually  a  scheduled  deadline  for 
completion. 

Supporting  Level.  This  level  is  analogous  to  office 
instructions  that  detail  how  internal  organizational  processes  are  to 
work.  For  example,  an  office  instruction  may  define  what  types  of 
information  may  be  sent  to  higher  headquarters,  under  whose 
signature,  and  under  what  circumstances.  In  Program  Offices 
organized  around  a  matrix,  instructions  may  exist  that  define  how 
individual  projects  under  the  control  of  the  PM  are  to  be  handled  in 
terms  of  management  team  make-up,  contract  change  review 
procedures,  etc.  Figure  16  illustrates  the  supporting  level. 

The  supporting  level  also  acknowledges  the  fact  that  the  PM  is 
not  only  a  manager  of  programs  but  is  also  a  leader  of  a  team  of 
individuals.  As  the  military  becomes  more  technologically  oriented 
with  increased  reliance  on  technical  expertise,  and  as  functional  staff 
specialization  increases,  the  military  manager  must  rely  more  and 
more  on  influence  in  lieu  of  simple  coercion  via  the  hierarchical 
authority  of  position  within  the  organization.  This  implies  the  use  of 
indirect  techniques  of  influencing  an  individual's  behavior  such  as 
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Figure  16 


Supporting  Level 


group  persuasion  and  an  emphasis  on  group  goals  (Harrison,  1978 
578).  As  a  leader,  the  PM  must  ensure  that  the  Program  Office 
understands  and  supports  the  decision  to  be  implemented.  To 
accomplish  that,  the  PM  must  use  coordination  and  compromise  in 
order  to  achieve  consensus. 

System  Structure.  The  structure  of  ODAM  must  account  for 
the  complexity  of  the  interface  between  the  four  acquisition  sub- 
processes  and  the  iterative  nature  of  decision  making  situations  and 
the  problem-solving  activities  undertaken  to  resolve  them  The 
ODAM  is  discussed  below  in  terms  of  inputs,  process,  and  outputs 
Inputs  The  decision  analysis  cycle’s  most  significant 

possible  contribution  is  its  ability  to  generate  quality  information 
that  can  be  brought  to  bear  on  complex  problems  in  an  uncertain 
world.  The  cycle  forces  the  organization  to  explicitly  deal  with  the 
following 

1.  Definition  of  the  problem, 
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2.  Determination  of  desired  outcomes, 

3.  Generation  of  possible  alternative  course  of 
actions  that  might  lead  to  those  outcomes, 

4.  Estimation  of  the  values  the  organization  places 
upon  each  of  those  outcomes, 

5.  Assessment  of  the  organization's  utility  function, 
and 

6.  Assessment  of  the  environmental  uncertainties 
surrounding  the  decision. 

The  challenge  facing  the  organization  is  to  ensure  that  its 
structure  and  managerial  processes  provide  the  raw  material 
necessary  to  allow  the  decision  analysis  cycle  to  work  This  is 
especially  true  when  determining  values  and  utilities.  Decision 
analysis  theory  offers  several  techniques  for  assessing  the  utilities  of 
individuals,  but  cannot  do  the  same  for  organizational  groups. 

All  large  organizations  create  plans  of  one  kind  or  another 
Plans  that  lay  out  strategic  intentions  for  the  organization  are  likely 
candidates  for  providing  utility  and  value  information  The 
organization  that  accepts  decision  analysis  and  implements  the 
ODAM,  would  be  required  to  consider  these  requirements  and 
possibly  modifying  its  procedures  for  preparation  of  planning 
documents  to  ensure  that  specific  statements  regarding  utility  and 
value  would  be  included. 

While  the  quality  of  these  planning  statements  may  lag  in  the 
early  stages  of  the  implementation,  as  the  ODAM  framework 
matures  within  the  organization  and  individual  participating 
managers  become  more  familiar  with  its  usage,  their  experience  will 


enrich  its  application.  As  time  progresses,  the  requirements  of  the 
decision  analysis  process  will  likely  creep  into  many  facets  of  the 
organization's  ongoing  operations.  Ultimately,  the  Program  Manager 
or  analyst  using  ODAM  should  encounter  few  significant  deficiencies 
in  the  information  available. 

Process.  The  ODAM  process  builds  upon  the  Japanese 
'decision  making'  approach  discussed  by  Drucker  (1974:  470).  Drucker 
summarizes  that  approach  as: 

1.  Focusing  on  what  the  decision  is  all  about, 

2.  Bringing  out  dissenting  opinions, 

3.  Focusing  on  alternatives  rather  than  the  'right 
solution',  and 

4.  Implementing  the  decision  after  consensus  is 
reached  ('selling'  the  decision  thus  being 
unnecessary). 

In  ODAM,  this  translates  to  an  institutionalization  of  a  process 
of  complete  and  thorough  coordination  throughout  the  organization 
The  first  purpose  of  the  coordination  is  to  bring  out  a  range  of 
opinions  from  different  segments  of  the  organization,  thus  enriching 
the  information  available  to  the  decision  makers.  The  second 
purpose  of  the  coordination  process  is  to  promulgate  a  meaningful 
compromise  among  the  various  elements  within  the  organization 
having  a  stake  in  the  decision.  The  result  of  the  coordination 
process,  after  the  compromise  stage,  is  an  organizational  consensus 
about  the  nature  of  the  problem  and  the  decision  to  be  made,  the 
alternatives  available  for  action,  how  the  alternatives  are  evaluated. 
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and  the  resulting  conclusions  with  regards  to  the  optimum,  or  most 
satisfactory,  alternative 

O’  tputs  Both  Drucker  and  Woolsey  emphasize  that  a 
decision  is  meaningless  if  it  is  not,  or  cannot,  be  implemented 
Therefore,  the  output  portion  of  ODAM  addresses  the  need  for  the 
organization  to  ensure  the  'do-ability'  of  the  course  of  action  decided 
upon. 

The  process  portion  of  the  ODAM,  wniie  enhancing  the 
organization's  ability  to  deal  with  the  political  arena,  does  not 
guarantee  success.  Concise  documentation  of  the  decision  making 
process,  from  its  genesis  to  conclusion,  should  be  available  during  and 
after  the  process.  Such  documentation  should  facilitate 
communication  during  the  coordination  activities  and  provide  for 
control  of  the  implemented  action  (Raiffa,  1968.  269). 

The  result  of  concise  documentation  and  effective  control  will 
then  be  the  best  opportunity  for  successful  implementation  of  the 
organization's  chosen  course  of  action. 

Evaluation 

The  ODAM  outlined  above  must  be  operationalized'  in  the 
context  of  unique  organizational  settings  -  for  instance  the  Air  Force 
acquisition  management  process.  Its  overall  prescriptions  and  its 
structured  framework  must  be  reflected  in  organizational  issues 
ranging  from  organizational  design  to  the  use  of  consultants  All 
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these  aspects  must  be  dealt  with  and  resolved  prior  to  exercising  the 
ODAM  if  satisfactory  results  are  to  be  obtained. 

The  organization  must  be  designed  to  complement  the 
coordination  activities  dictated  by  the  model.  The  organizational 
structure  and  definition  of  responsibilities  must  be  aligned  in  a 
manner  consistent  with  the  desire  to  reach  consensus  through 
compromise.  Likewise,  without  efficient  communication  channels 
within  the  organization,  to  all  parts  of  the  organization,  effective 
coordination  will  not  be  possible.  If  an  organization  lacks  these 
characteristics,  the  effort  of  implementing  ODAM  will  result  in 
frustration  and  probably  failure. 

All  organizations  create  information.  To  effectively  implement 
the  ODAM,  the  formal  procedures  for  preparing  organizational  plans 
must  look  forward  to  the  fact  that  those  plans  might  someday  be 
required  to  support  the  requirements  of  decision  analysis  activities. 
Specifically,  the  plans  must  include  reference  to  the  relative  values 
the  organization  assigns  to  various  future  conditions  -  both  in  its 
own  situation  and  the  state  of  the  external  environment. 
Management  information  systems  (MIS)  must  also  be  designed  to 
provide  all  the  information  possible  to  support  future  decision 
analysis  exercises.  MIS  output  can  prove  beneficial  when  defining 
alternatives,  assessing  uncertainties,  and  supporting  valuations. 

The  ODAM  will  be  used  by  people  -  in  this  study  the  acquisition 
Program  Manager  and  the  program  management  team  has  been  the 
focus  of  the  discussions.  The  requirements  of  the  model  should 
influence  both  job  definition  and  personnel  placement.  Any  tool,  no 
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matter  how  inherently  effective,  cannot  succeed  if  employed  by 
misplaced  or  under-trained  personnel. 


Summary 

The  Organizational  Decision  Analysis  Model,  summarized  in 
Figure  17,  offers  the  opportunity  to  implement  into  the  Air  Force 
acquisition  process  a  rational  and  coherent  decision 
making  framework  for  use  by  the  acquisition  Program  Manager 
While  based  upon  a  process  that  is  inherently  not  intended  for 
organizational  implementation,  the  ODAM  makes  allowances  for  the 
relevant  differences  in  the  decision  making  situation  so  that  the 
benefits  of  rational,  consistent  and  defensible  decision  making  remain 
intact. 

From  the  perspective  of  the  Program  Manager,  the  decision 
analysis  cycle  in  ODAM's  managerial  level  provides  the  necessary 
structure  to  information  gathering  activities  that  can  be  lacking 
when  attempting  to  manage  problems  that  may  extend  over  several 
acquisition  system  sub-processes.  Similarly,  the  supporting  level 
addresses  the  need  for  the  PM  to  get  an  organizational  consensus 
from  within  the  decision  making  unit  about  the  many  inputs  to  the 
decision  analysis  cycle.  It  is  the  supporting  level  that  builds  the 
foundation  for  the  Program  Manager  to  eventually  acquire  a 
consensus  from  the  organizational  as  a  whole  at  the  organizational 
level  The  organizational  level  itself  highlights  for  the  PM  critical 
considerations  with  respect  to  the  environment  and  systemic  inputs 
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and  output  requirements  that  may  be  outside  the  PM‘s  immediate 
ability  to  control.  ODAM  is  a  framework  for  consistent,  explicit  and 
effective  management  decision  making  by  the  acquisition  Program 
Manager. 

Next,  Chapter  V  will  analyze  the  Air  Defense  Fighter 
Competition  by  comparing  that  recent  acquisition  decision  to  the 
ODAM.  The  analysis  will  proceed  as  described  in  Chapter  III.  The 
results  of  the  study  will  be  summarized  in  Chapter  VI  and 
conclusions  drawn. 


V  Comparative  Analysis 

Introduction 

The  purpose  of  this  chapter  is  to  analyze  the  acquisition 
Program  Manager's  decision  support  documentation  used  during  the 
Air  Defense  Fighter  (ADF)  Competition.  This  documentation  was 
reviewed  as  described  in  Chapter  III  and  the  major  events  during  the 
decision  making  process  are  highlighted.  Next,  the  documentation 
was  categorized  and  compared  to  the  Organizational  Decision  Analysis 
Model  (ODAM)  developed  in  Chapter  IV.  The  results  of  the 
categorization  were  then  analyzed  for  evidence  that  the  ODAM  might 
be  appropriate  for  use  in  the  DOD  Weapon  System  Acquisition  (WSA) 
process  and  that  the  primary  WSA  sub-processes  could  support  the 
needs  of  the  decision  analysis  methodology.  As  described  in  Chapter 
III,  the  comparative  analysis  focused  on  information  sources, 
communications  channels  and  management  actions  during  the  ADF 
decision  process  as  documented  by  the  program  management  team. 


Historical  Overview 

As  discussed  in  the  literature  review,  several  changes  and 
events  affecting  the  WSA  process  had  occurred  during  the  middle 
1980's.  These  changes  were  the  passage  of  the  Competition  in 
Contracting  Act  (CICA)  of  1984,  the  Goldwater-Nichols  DOD 
Reorganization  Act  of  1986,  the  Graham-Rudman-Holhngs  Balanced 
Budget  and  Emergency  Deficit  Control  Act  and  the  publication  of  the 


report  of  the  President's  Blue  Ribbon  Panel  on  Defense  Management. 
These  changes  and  events  contributed  to  a  situation  that  required  a 
series  of  acquisition  decisions  on  the  part  of  Air  Force  management. 
There  was  a  prevailing  sense  that  DOD  acquisition  management  was 
inefficient  and  that  competition  in  contracting  was  a  cure  for  that 
problem.  At  the  same  time,  there  was  considerable  pressure  to 
reduce  the  DOD  budget. 

On  2  April  1985,  Thomas  V.  Jones,  Chairman  of  the  Board  and 
Chief  Executive  Officer  of  Northrup  Corporation,  sent  a  letter  to 
Secretary  of  the  Air  Force  Verne  Orr  and  Air  Force  Chief  of  Staff 
Charles  A.  Gabriel  (Jones,  1985).  This  letter  was  an  unsolicited 
proposal  to  develop  and  sell  to  the  Air  Force  396  F-20  fighter 
aircraft. 

This  letter  initiated  the  first  of  three  phases  of  Air  Force 
activity  to  arrive  at  a  resolution  to  the  problem  posed  by  Northrup 's 
unsolicited  proposal  (i.e.  what  action  should  the  Air  Force  take  with 
respect  to  the  F-20).  In  the  first  phase,  the  Air  Force  was  required 
to  respond  to  the  Northrup  proposal.  The  purpose  of  the  response 
was  to  notify  Northrup  whether  the  Air  Force  had  decided  to  accept 
the  proposal. 

The  letter  outlined  a  proposal  to  deliver  396  F-20  aircraft  in  a 
four-year  fixed  price  multi-year  procurement,  with  a  Fiscal  Year 
(FY)  1986  unit  procurement  price  of  $15.0  million  (then  year  dollars). 
The  first  USAF  F-20  was  to  be  delivered  24  months  from  go-ahead 
and  the  final  aircraft  in  FY91.  The  follow-on  support  was 
guaranteed  at  a  fixed  price  of  $475  per  flying  hour  (FY85  dollars). 


The  proposal  letter  mentioned  that  Northrup's  F-20 
development  was  originally  intended  to  meet  the  "Government's 
expressed  need  for  a  modern  fighter  that  could  help  friendly  nations 
achieve  the  maximum  level  of  national  security  without  overtaxing 
their  resources"  (Jones,  1985).  The  original  intent  of  the  F-20 
program  was  to  meet  a  Carter  administration  effort  (circa  1978)  to 
produce  an  inexpensive  lightweight  fighter  to  support  Foreign 
Military  Sales  (FMS).  The  Reagan  administration  reversed  this  policy 
in  1982  when  it  began  selling  F-16  and  F-15  aircraft  under  FMS 
guidelines.  When  no  sales  were  realized  after  five  years  of 
development  and  two  of  three  F-20  prototypes  crashed,  Northrup 
changed  its  marketing  approach.  Northrup's  proposal  to  the  Air 
Force  was  coordinated  with  an  attempt  to  influence  the  Reagan 
administration  and  the  Congress  that  the  U.  S.  Air  Force,  originally 
an  unintended  client,  should  buy  the  plane.  The  plane  was  being 
"...  equated  with  the  virtues  of  competition"  (Gordon,  1985:  1608). 
Northrup's  approach  appeared  rational  given  the  recent  passage  of 
the  CICA  and  the  threat  of  Graham-Rudman  deficit  control  budget 
cuts. 

CEO  Jones  stated  in  his  letter  of  2  April  1985  that: 

the  Air  Force  now  [had]  the  opportunity  for  an  F- 
20/F-16  complementary  force,  acquired  and  apportioned 
with  the  benefit  of  competition,  that  will  reduce 
procurement  and  operating  costs  and  give  the  Air  Force 
more  flexibility  in  managing  the  size  and  mix  of  its 
fighter  force  within  current  and  projected  budget 
constraints  (Jones,  1985). 


The  proposal  went  on  to  stress  the  cost  advantages  of  the  F-20 
as  compared  to  the  F-16  while  pointing  out  the  relative  performance 
strengths  of  the  F-20  for  certain  missions.  The  continental  U  S.  air 
defense  role  was  one  of  these  missions. 

A  Northrup  proprietary  Proposal  Summary,  attached  to  the 
letter,  included  a  comparison  of  F-20  force  effectiveness  compared  to 
the  F-16.  The  proposal  emphasized  logistical  measures  of 
effectiveness.  The  ultimate  measure  of  effectiveness  was  described  as 
capability  when  operating  as  a  force.  The  proposal  listed  contributing 
factors  as  high  reliability,  low  maintenance,  low  mean  time  to  repair 
and  quick  turnaround  times. 

A  second  proprietary  attachment  was  titled  Program 
Acquisition.  This  document  was  a  very  brief  review  of  the  price 
basis  of  Northrup 's  proposal  (weapon  system  acquisition  proposals  are 
usually  cost  based).  The  document  covered  the  multi-year 
procurement  quantities,  aircraft  flyaway  prices,  initial  support  prices 
with  conditions  and  assumptions,  an  economic  price  adjustment 
clause,  progress  payment  provision  requirements,  Northrup’s  follow- 
on  support  guarantee,  warranty  and  performance  guarantees,  and  a 
program  cancellation  provision.  Specific  details  of  these  two 
documents  cannot  be  included  here  because  of  their  proprietary 
nature. 

Within  the  Aeronautical  Systems  Division  (ASD),  responsibility 
for  the  proposal  evaluation  was  assigned  to  tne  Deputy  Commander 
for  Tactical  Systems  (ASD/TA).  Responsibility  was  further  delegated 
to  ASD/TA  A,  the  Fighter/ Attack  System  Program  Office  (SPO).  The 


Vice  Commander  (ASD/CV)  was  made  the  executive  point  of  contact 
within  ASD. 

On  2  May  1985,  ASD/CV  signed  a  letter  to  the  Deputy  Chief  of 
Staff  (DCS)  for  Systems  at  the  Headquarters  of  Air  Force  Systems 
Command  (AFSC/SD)  transmitting  the  completed  evaluation  of  the 
Northrup  unsolicited  proposal.  The  evaluation  addressed  the 
following  points  which  are  relevant  to  acquisition  decision  making. 

1.  Without  a  stated  user  requirement,  the  SON 
from  the  requirements  sub-process,  evaluation  of 
the  Northrup  proposed  configuration  was  not 
possible. 

2.  The  proposal  did  not  comply  with  Federal 
Acquisition  Regulation  (FAR)  requirements  as 
defined  in  the  contracting  sub-process. 

3.  The  proposal  did  not  include  guarantees  for 
performance,  reliability,  maintainability, 
availability,  or  required  modifications  which  were 
important  considerations  during  the  acquisition 
management  sub-process. 

Along  with  the  evaluation  report  was  a  proposed  letter  to 
Northrup  stating  that  while  the  Air  Force  was  pleased  to  receive  the 
unsolicited  proposal,  it  could  not  be  accepted  because  of  the  deviations 
from  the  FAR  and  other  regulations.  Also,  the  letter  stated  that  the 
proposal  lacked  Air  Force  approved  specifications  and  a  complete  test 
plan. 

At  this  time  program  management  documentation  changed  its 
focus  and  the  second  phase  of  activity  began.  Almost  from  the 
beginning,  concerns  arising  from  the  Northrup  unsolicited  proposal 
took  a  broader  perspective  than  the  evaluation  and  response  in  the  2 


May  1985  Air  Force  response  to  Northrup.  The  issue  was  generally 
seen  in  the  context  of  overall  tactical  aircraft  competition  and  the 
possible  effects  of  competition  on  force  structure.  The  second  phase 
involved  activities  directed  toward  a  ’limited  competition'  in  which 
the  F-20  would  be  one  of  the  competitors..  At  this  point,  the  key 
players  constituted  ASD/CV,  ASD/TA  and  staff,  ASD/CC  (the 
Commander)  and  his  Chief  of  Staff  (CS),  AFSC/SD,  ASD/YZ  (the 
Deputy  Commander  for  Propulsion),  and  ASD/PM  (the  Deputy 
Commander  for  Procurement).  As  early  as  15  April  1985,  the 
following  issues  were  discussed  at  HQ  AFSC:  (1)  the  strong  possibility 
that  the  F-20  procurement  would  be  added  to  the  FY87  Program 
Objective  Memorandum  (POM)  and  (2)  that  the  Air  Force  would  push 
for  a  "head-to-head  competition"  between  the  F-16  and  the  F-20  By 
15  May  1985,  meetings  at  HQ  AFSC  and  the  Air  Staff  with 
representatives  from  ASD  were  held  to  discuss  possible  Air  Force 
strategies  for  the  competitive  procurement  of  tactical  fighters. 

On  22  August  1985  a  memo  was  sent  to  ASD/CC  proposing  an 
F-20  Program  Plan  in  response  to  direction  received  in  an  ASD/CC 
memo  of  27  July  1985  to  begin  such  planning  .  The  program  plan  is 
the  first  documented  example  of  ASD  assuming  the  role  of  'problem 
solver'  and  formulating  a  definite  approach.  The  plan  consisted  of  F- 
20  POM  cost  information,  F-20  schedules,  a  competition  evaluation 
plan  and  schedule  for  the  Air  Defense  competition  (AD-X),  F-20  SPO 
approved  schedule  and  costs  estimates,  and  a  review  of  FAR  waivers 
and  deviations. 
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The  proposed  F-20  Program  Plan  was  the  culmination  of  the 
second  phase  of  Air  Force  activity.  Notable  in  the  plan,  in  terms  of 
organizational  decision  making,  was  an  alternative  approach  to  the 
Air  Defense  Fighter  competition  based  upon  the  Dual  Role  Fighter 
(DRF)  Competition  of  1984  (F-16F  versus  the  F-15E).  The  action  plan 
described  the  proposed  organization  of  the  evaluation  team,  stated 
how  evaluation  methodology  was  to  be  determined,  described  how 
the  evaluation  plan  should  be  developed,  and  stated  how  the 
evaluation  should  be  implemented.  Preliminary  schedule  projections 
were  also  made.  These  reflected  dates  for  the  receipt  of  direction, 
the  release  of  a  Request  for  Proposal  (RFP),  the  subsequent  receipt  of 
contractor  proposals,  the  formal  evaluation  of  those  contractor 
proposals,  the  conduct  of  Operational  Test  &  Evaluation  (0T8.E),  final 
decision  and  the  award  of  a  contract. 

Little  came  of  this  early  effort  to  narrow  the  programmatic 
options.  More  and  more  the  organization’s  concerns  focused  on  how 
to  comply  with  Congressional  direction  to  proceed  with  a  tactical 
fighter  competition  in  FY  86  while  also  complying  with  other  existing 
laws  and  regulations.  Schedule  was  the  primary  constraint 
identified.  The  chief  components  of  that  constraint  were  F-20 
Development  Test  &  Evaluation  (DT&E)  and  10TS.E  requirements,  and 
the  many  FAR  deviations  and  waivers  necessary  to  meet  the  early 
contract  award  date  proposed  by  Northrup  and  alluded  to  by 
Congress. 
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On  September  9,  1985,  ASD/CC  sent  a  message  to  the 
Commander  of  Systems  Command  (AFSC/CC)  identifying  several 
concerns.  He  stated  that 

. . .  while  there  is  lots  of  talk  about  introducing  the  F-20 
aircraft  into  the  USAF  inventory,  there  doesn’t  seem 
to  be  any  formal,  structured  dialog  on  the  subject 

it  concerns  me  that  such  a  setting  is  a  great  one  for 
getting  the  wrong  decisions  made . . .  frustrating  as  our 
normal  process  can  be,  it  at  least  has  a  formal  process 
within  which  these  basic  program  decisions  are  made 
(McMullen,  1985)  [Emphasis  added]. 

Discussions  about  how  to  structure  a  competition  similar  to  the 
DRF  lasted  only  briefly.  In  response  to  a  22  August  1985  Program 
Decision  Memorandum  (PDM)  from  the  Deputy  Secretary  of  Defense 
(DEPSECDEF),  a  full  and  open  competition  for  an  Air  Defense  fighter 
replaced  the  'DRF-like  comparison’  between  the  F-16  and  F-20  This 
action  initiated  the  third  phase  of  activity. 

In  a  16  September  1985  memorandum  to  the  DEPSECDEF,  the 
Secretary  of  the  Air  Force  outlined  the  Air  Force  "...  plan  to  proceed 
with  an  unrestricted  competition  for  an  Air  Defense  aircraft"  (Orr, 
1985).  A  point  paper  attached  to  that  memorandum  noted  that  this 
action  was  consistent  with  the  22  August  1985  Program  Decision 
Memorandum  (PDM)  and  would  use  full  source  selection  procedures 
required  by  AFR  70-15,  Source  Selection  Procedures.  The  point  paper 

also  stated  that  Air  Force  efforts  would  be  focused  upon  developing  a 
Request  for  Proposal  (RFP),  a  Source  Selection  Plan  (SSP),  and  other 
procurement  related  documents.  The  proposed  schedule  of  events 
leading  to  contract  award  was  listed  as: 


1.  RFP  release  -  January  1986, 

2.  Contractor  responses  due  -  March  1986, 

3.  Evaluation  complete  -  July  1986, 

4.  Contract  Award  -  October  1986. 

Activity  after  mid-September  was  almost  totally  directed 
toward  three  ends.  First,  a  Program  Management  Directive  (PMD) 
was  required  before  the  RFP  could  be  released  to  potential  sources. 
The  final  PMD  (Number  PMD  5268/27139F)  was  signed  by  AF/RDQ 
and  issued  to  the  Major  Commands  on  21  November  1985.  The 
Executive  Summary  of  the  PMD  stated  that 

. . .  this  Program  Management  Directive  (PMD)  provides 
direction  for  the  full  scale  development,  procurement,  and 
support  of  Air  Defense  (AD)  aircraft.  The  aircraft  will  be 
used  primarily  for  Strategic  Air  Defense  of  the 
continental  United  States  (DAF,  1985c). 

The  PMD  also  referenced  the  Tactical  Air  Forces  (TAF) 

Statement  of  Operational  Need  (SON)  applicable  to  this  program. 

However,  the  SON  at  this  time  was  still  in  the  validation  process  and 

was  listed  as  “for  comment"  only.  The  TAF  SON  (Number  310-85) 

was  validated  on  7  January  1986.  The  SON  was  critical  to  the 

contracting  approach  since  a  System  Requirements  Document  (SRD) 

was  to  be  issued  with  the  RFP  instead  of  a  system/item  specification. 

The  SRD  would  be  based  upon  the  requirements  stated  in  the 

validated  SON. 

After  issuing  a  draft  RFP  to  industry  and  interested 
government  organizations,  the  final  ADF  RFP  was  issued  on  24 
March  1986.  The  RFP  was  sent  to  McDonnell  Douglas  Corporation, 


Boeing  Military  Airplane  Company,  General  Dynamics  Corporation, 
and  Northrup  Corporation.  Proposals  were  due  to  the  Air  Force  for 
evaluation  by  9  May  1986.  This  allowed  only  45  days  for  the 
contractors  to  prepare  their  proposals.  Normally,  contractors  are 
allowed  60  days.  The  accelerated  schedule  was  an  indication  of  the 
Air  Force  attempt  to  meet  the  Congressionally  mandated  timetable 
for  a  decision. 

The  prioritized  areas  of  evaluation  were  (in  descending  order  of 
importance)  Mission  Effectiveness,  Technical,  Life  Cycle  Cost,  and 
Logistics/Management.  The  contractors'  proposals  were  to  be 
assessed  using  the  following  criteria  (each  ranked  equally):  soundness 
of  approach,  understanding  the  requirements,  compliance  with 
requirements,  and  past  performance  (ASD,  1986:  Section  M). 

Ultimately,  only  General  Dynamics  and  Northrup  submitted 
proposals.  Northrup  submitted  a  proposal  for  its  F-20.  General 
Dynamics  submitted  two  proposals.  One  proposal  was  for  an  F-16SC 
and  the  other  was  for  a  retrofit  program  to  modify  existing  F-16A/B 
aircraft.  The  proposed  F-16SC  (Specially  Configured)  was  essentially 
an  F-16C  without  the  high  cost  avionics  and  radar  capabilities 
necessary  for  a  front-line  tactical  multi-role  aircraft.  The  changes 
proposed  by  General  Dynamics  as  part  of  their  F-16SC  proposal  were 
intended  to  place  it  in  better  price  competition  with  the  F-20  and  to 
acknowledge  that  the  requirements  of  the  AD  mission  were  less 
rigorous  than  other  tactical  missions  (e  g.  close  air  support,  defensive 
counter  air,  etc.)  to  which  the  F-16  was  assigned. 


On  31  October  1986,  the  Secretary  of  the  Air  Force  held  a  press 
conference  at  which  time  he  announced  the  winner  of  the  ADF 
Competition.  The  winner  was  the  General  Dynamics  proposal  to 
modify  existing  F-16A/B  aircraft. 

Analysis  of  Selected  Decision  Support  Documentation 

As  described  in  Chapter  III,  the  program  management 
documents  assembled  by  the  Program  Manager  of  the  ADF 
Competition  were  categorized  in  accordance  with  the  content  analysis 
plan.  The  plan  was  to  compare  subcategories  (variables)  most 
relevant  to  organizational  decision  making.  The  data  generated  by 
the  content  analysis  were  then  statistically  analyzed  to  determine  if 
any  correlation  existed  between  pairs  of  subcategories.  As  discussed 
in  Chapter  III,  the  indicator  chosen  for  possible  correlation  was  the 
contingency  coefficient.  The  results  of  the  analysis  are  summarized 
below 

The  ADF  Competition  was  initiated  on  2  April  1985  by  the 
receipt  of  the  unsolicited  proposal  from  Northrup  and  the  last 
decision  was  finalized  on  31  October  1986  when  General  Dynamics 
was  announced  as  the  winner  of  the  competition.  The  frequency 
distribution  of  documents  over  that  time  period  followed  a  pattern 
which  peaked  mid-way  through  the  cycle  with  much  less  activity  at 
both  the  beginning  and  end.  The  vast  majority  of  documented 
activity  (58.8$)  occurred  during  the  three  month  time  period  from 
October  until  December  1985.  This  coincides  with  the  efforts  of  the 


Air  Force  to  issue  the  PMD  (21  November  1985)  for  the  Air  Defense 
Fighter  program  as  well  as  to  validate  the  Strategic  Air  Defense  TAF 
SON  (7  January  1986).  From  January  thru  March,  activity  abated 
in  frequency  but  was  still  important  because  the  efforts  of  the  ADF 
program  management  team  were  now  directed  primarily  at  the 
single  task  of  issuing  the  RFP. 

In  the  time  period  following  the  RFP  release  in  March  1986, 
documentation  was  even  less  frequent.  Only  9.55%  of  all  documents 
fell  within  the  seven  month  period  from  April  through  October  1986. 
For  most  of  this  period,  the  program  management  team  was 
involved  in  source  selection  tasks,  which  are  treated  as  sensitive  and 
not  the  subject  of  communication  activity.  After  the  contractor 
proposals  were  received  on  9  May  86,  documentation  was  relatively 
infrequent  thru  the  end  of  October  1986  at  which  point  the 
documentation  effort  ceased. 

During  the  entire  time  period  covered  by  the  documentation, 
each  document  was  categorized  as  to  its  flow  within  or  between 
organizations.  Most  documents  were  either  transmitted  within  the 
organization  (Internal:  31.66%)  or  were  received  by  the  organization 
from  some  other  originating  office  (Incoming:  31.66%).  Referring  to 
Figure  12  in  Chapter  IV,  the  organization's  boundaries  were  shown  to 
be  the  primary  organizational  elements  involved  in  the  ADF  source 
selection.  Documents  sent  out  of  the  organization  accounted  for  13.9% 
and  documents  in  the  NA  category  (e  g.  magazine  articles,  etc.) 
accounted  for  22.01%.  Only  two  documents,  or  .77%  were  of  unknown 
origin.  The  contingency  coefficient  for  the  comparison  of  document 
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flow  versus  time  was  .547.  The  p-value  was  .0023  which,  at  a  .05 
level  of  confidence,  indicates  that  the  flow  of  documents  was 
correlated  to  the  time  period  within  the  decision  process.  In  other 
words,  the  make-up  of  each  individual  flow  category  was  dependent 
upon  the  month  during  the  cycle  in  a  manner  dissimilar  to  the 
overall  frequency  of  documents  over  time.  Therefore,  the  level  of 
individual  document  flow  directions  can  be  identified  more  with  one 
period  than  another.  Depending  on  the  month  during  the  cycle,  the 
Program  Manager  would  have  been  relying  on  different 
communication  (i.e.  internal  or  outgoing)  channels  to  meet 
management  objectives. 

The  breakdown  of  the  documents  by  the  problem  solving  steps 
is  discussed  next.  The  problem  definition  step  accounted  for  16.03%  of 
the  documents.  Alternatives  identification  and  gathering  data 
totalled  16.41%  and  44.27%  respectively.  Evaluation  accounted  for 
16.41%  while  decision  related  documents  were  only  3.82%  of  the  total. 
The  data  gathering  step  was  the  most  frequent  activity  by  a  factor 
of  almost  three.  As  mentioned  before,  while  the  volume  of  decision 
related  documents  was  low,  their  importance  to  the  progress  of  the 
overall  acquisition  process  and  its  ultimate  success  was  critical. 

The  results  of  the  contingency  table  analysis  comparing  the 
problem  solving  steps  versus  time  is  shown  in  Table  2. 

While  only  the  problem  definition  step  was  determined  to  be 
related  to  time  at  a  confidence  level  of  .05  (p-value  =  .0001),  the  data 
gathering  step  was  close  with  a  p-value  of  .054.  Also,  both  the 
identification  of  alternatives  step  and  the  evaluation  step  had 


Table  2 

Contingency  Coefficient/p-value:  Time  vs.  Problem  Solving  Steps 
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Step 

Problem  Definition 

.432/0001 

ID  Alternatives 

.308/. 0702 

Gather  Data 

.314/0540 

Evaluation 

,308/0702 

Decision 

.255/  4385 

BOLDFACE:  Statistically  significant  at  .05 

p-values  of  .0702.  Only  the  decision  step  was  found  to  exhibit  little 
relationship.  Its  p-value  was  .4385.  Therefore,  the  problem  solving 
step  documents  were  dependent  on  the  time  of  the  cycle.  The 
observed  frequency  table  data  substantiates  this.  During  the  October 
1985  through  January  1986  period,  54.76$  of  the  problem  definition 
documents  were  generated.  Another  19.05$  were  generated  in  April 
1985  and  16  67$  in  September  1985.  In  the  period  from  February 
through  October  1986,  no  problem  definition  documents  were 
generated.  Therefore,  the  generation  of  problem  definition  related 
documentation  closely  followed  the  timing  of  the  major  documents  in 
the  decision  making  process:  receipt  of  the  unsolicited  proposal  in 
April  1985,  the  work  to  issue  the  PMD  (November  1986)  and  to 
validate  the  SON  (January  1986),  and  the  effort  to  prepare  an  issue 
the  draft  RFP  (January  1986).  This  statistically  significant 
relationship  is  exactly  what  would  be  expected  to  occur. 


-values  of  .0702.  Only  the  decision  step  was  found  to  exhibit  little 
relationship.  Its  p-value  was  .4385.  Therefore,  the  problem  solving 
step  documents  were  dependent  on  the  time  of  the  cycle.  The 
observed  frequency  table  data  substantiates  this.  During  the  October 
1985  through  January  1986  period,  54.761  of  the  problem  definition 
documents  were  generated.  Another  19.05$  were  generated  in  April 
1985  and  16.67$  in  September  1985.  In  the  period  from  February 
through  October  1986,  no  problem  definition  documents  were 
generated.  Therefore,  the  generation  of  problem  definition  related 
documentation  closely  followed  the  timing  of  the  major  documents  in 
the  decision  making  process,  receipt  of  the  unsolicited  proposal  in 
April  1985,  the  work  to  issue  the  PMD  (November  1986)  and  to 
validate  the  SON  (January  1986),  and  the  effort  to  prepare  an  issue 
the  draft  RFP  (January  1986).  This  statistically  significant 
relationship  is  exactly  what  would  be  expected  to  occur. 

The  documents  were  also  organized  according  to  the  acquisition 
sub-process  they  were  associated  with.  Considering  all  documents, 
the  requirements  sub-process  accounted  for  10.31$  and  the  budget 
sub-process  was  8,02$  of  the  total.  The  acquisition  management 
sub-process  was  much  more  frequent  at  44.27$  while  the  contracting 
sub-processes,  at  50.0$,  was  the  most  frequent  (Note:  the 
percentages  need  not  total  to  100$  since  a  single  document  might 
support  more  than  one  sub-process).  Table  3  shows  the  resulting 
contingency  table  analysis  of  the  four  sub-processes  versus  time. 

The  contingency  coefficient  for  the  requirements  sub-process 
indicated  that  it  was  not  significantly  related  to  the  distribution  of 


Table  3 

Contingency  Coefficient/p-value:  Sub-Processes  vs.  Time 


Requirements  Budget  Acq  Mgt  Contracting 

286/  1801*  .341/0108  .483/  0001  363/  0023 

BOLDFACE:  Statistically  significant  at  .05 
*•  Invalid  test  due  to  expected  frequencies  <  1 


documents  over  time.  This  result  was  not  expected  since  it  implies 
that  the  requirements  sub-process  was  not  more  prevalent  in  the 
early  months  of  the  decision  making  process  as  would  normally  be 
the  case.  This  is  especially  so  since  it  was  shown  earlier  that 
problem  definition  documents  were  found  to  be  related  to  time  and 
they  are  analogous  to  defining  requirements.  Based  on  the  detailed 
observed  frequency  tables,  the  10.31$  of  all  documents  relating  to  the 
requirements  process  were  all  observed  prior  to  February  1986.  The 
interest  and  concern  of  the  managers  toward  the  PMD  and  SON  was 
evident  but,  once  the  organization  had  documented  its  requirements, 
that  documented  concern  vanished.  Also,  once  the  requirements 
were  reflected  in  the  SRD  and  the  RFP,  continued  reference  to  these 
documents  was  not  necessary  (disregarding  any  possible  updates). 
Inferences  based  upon  the  data  in  the  later  months,  however,  must 
be  questioned  since  in  the  period  from  April  to  October  1986  all  cells 
exhibit  an  expected  frequency  of  less  than  1.0.  Thus,  although  not 


statistically  significant,  as  expected,  the  requirements  documents  do 
occur  in  the  appropriate  time  frame. 

Each  of  the  other  three  sub-processes  were  related  to  time  It 
was  expected  that  the  acquisition  management  sub-process  would  be 
related  to  time,  mirroring  the  initial  activity  in  April  1985,  the  peak 
at  the  end  of  calender  year  1985  coinciding  with  the  PMD,  and  the 
lull  after  March  1986  when  source  selection  activity  became 
dominant. 

Likewise,  the  contracting  sub-process  would  be  assumed  to  be 
related  to  time  for  similar  reasons.  The  unsolicited  proposal  in  April 
1985  was  also  a  contracting  action.  Activities  in  the  period  from 
October  through  December  1985  were  focused  on  the  preparation  of 
the  draft  RFP  to  be  released  in  January  1986.  After  the  RFP  was 
finally  released  on  24  March  1986  and  the  source  selection  began  on 
9  May  1986,  contracting  related  documents  would  become  less  visible. 

The  budget  sub-process  was  not  expected  to  be  related  to  time. 
The  ADF  program  management  documentation  does  not  show 
evidence  that  the  management  team  was  actively  involved  in  the 
POM  process.  In  fact,  most  budget  related  documentation  was 
information  provided  to  the  PM  to  facilitate  program  planning.  It 
must  be  assumed,  therefore,  that  budget  was  related  to  time,  not 
because  of  the  ADF  Competition  schedule  of  events  but  instead, 
because  of  the  overall  PPBS  schedule  of  events  in  the  Air  Force  as 
well  as  in  the  Congress. 

The  final  comparison  focusing  on  time  was  with  the  three 
communications  tasks:  information  search,  intra-organizational 


coordination,  and  extra-organizational  communication.  Table  4 
summarizes  the  results  of  the  contingency  table  analysis. 

Table  4 


Contingency  Coefficient/p-value:  Communication  Tasks  vs.  Time 


Information 

Intra-Org. 

Extra-Org. 

Search 

Coordination 

Coordination 

.253/. 4570 

.386/. 0003 

.355/. 0042 

BOLDFACE:  Statistically  significant  at  .05 

Both  of  the  coordination  activities,  within  the  organization  and 
outside  the  organization,  were  found  to  be  related  to  time.  As 
above,  75.35%  of  the  intra-organizational  coordination  came  in  the 
three  month  period  from  October  to  December  1985  and  6.85%  came 
in  the  early  phase  of  April  and  May  1985.  No  intra-organizational 
coordination  documents  were  found  in  the  months  after  June  1986. 
Again,  intra-organizational  coordination  appeared  linked  to  the  major 
decision  documents  of  the  unsolicited  proposal  evaluation,  the  PMD, 
the  SON,  and  the  RFP. 

Extra-organizational  coordination  had  a  distribution  similar  to 
the  intra-organizational.  In  the  October  to  December  1985  time 
period,  56.76%  of  the  external  coordination  seeking  documents  were 
found.  It  was  different  from  intra-organizational  coordination, 
however,  in  that  extra-organizational  coordination  was  skewed  away 
from  the  early  months  of  the  decision  process  toward  the  latter. 


External  coordination  did  not  really  begin  until  September  1985,  after 
the  PDM  was  issued.  Prior  to  that  time,  it  appears  that  program 
management  approached  the  ADF  issue  as  essentially  an  internal 
problem  and  that  any  request  for  outside  assistance  and  support  was 
premature. 

Information  search,  however,  was  found  to  be  not  related  with 
time.  While  information  search  exhibited  the  same  mid-cycle  peak 
as  above  (68.3%  in  October  thru  December  1985),  it  was  also  broader 
in  scope,  extending  from  September  1985  to  March  1986.  There  were 
no  information  search  documents  after  March  1986.  Information 
search  was  then  a  broader  subcategory  than  the  two  coordination 
activities.  The  information  search  documents  in  the  early  phase 
(April  to  May  1985)  coincide  with  the  intra-organizational 
coordination  associated  with  the  evaluation  of  the  unsolicited 
proposal.  The  information  search  documents  coming  in  the  period 
from  January  thru  March  1986,  however,  coincide  with  the  extra- 
organizational  coordination  associated  with  the  final  preparation  of 
the  RFP.  Overall,  information  search  documents  followed  the  general 
frequency  trend  of  all  the  documents.  Information  search  shared  the 
mid-period  peak  with  the  other  two  communication  processes. 
However,  information  search  documents  in  the  early  phase  were 
primarily  searching  internally  while  in  the  latter  phase  they  were 
searching  externally. 

The  preceding  analysis  of  the  subcategory  time  (Tables  2-4) 
revealed  the  following  about  the  ADF  Competition  decision  making 
process.  First,  although  problem  definition  documents  were 


significantly  correlated  with  time  and  were  shown  to  be  frequent 
early  in  the  process  as  would  be  expected,  the  frequency  of  the 
documents  supporting  the  requirements  sub-process  were  shown  to 
be  independent  of  time.  Since  normal  acquisition  procedures  dictate 
that  the  SON  be  validated  before  other  acquisition  management 
actions  occur  (e.g.  enter  a  new  program  into  the  POM,  issue  of  new 
PMD,  etc.),  the  fact  that  this  effort  was  initiated  by  an  unsolicited 
proposal  and  accelerated  by  the  Congressionally  directed  schedule 
probably  accounts  for  the  majority  of  this  result.  Also,  documents 
were  described  as  problem  solving  that  did  not  support  the 
requirements  sub-process.  There  were  also  problem  solving 
documents  in  the  acquisition  management  and  contracting  sub¬ 
processes  as  well. 

The  levels  of  document  flow  direction  were  also  found  to  be 
significantly  related  to  time.  Internal  documents,  for  instance, 
tended  to  be  more  frequent  in  some  months  than  others.  Similarly, 
mtra-organizational  coordination  was  also  found  to  be  significantly 
related  to  time.  These  results  underscore  the  importance  of  gaining 
an  internal  organizational  consensus.  Thus,  the  data  support  the 
proposition  that  in  the  WSA  process,  individual  decision  makers,  as 
defined  by  Howard,  are  the  exception  and  not  the  rule.  The 
complexity  of  the  programs  under  study  necessitate  elaborate 
coordination  review  cycles  of  information  products.  The  observed 
frequency  tables  show  this  to  be  particularly  so  in  the  unsolicited 
proposal  evaluation  phase  and  in  the  three  month  peak  period  just 
before  the  draft  RFP  release. 

133 


Extra-organizational  coordination  was  also  related  with  time, 
but  differed  from  intra-orgamzational  coordination  in  that  its 
observed  frequency  continued  after  the  October  to  December  1985 
peak  and  lasted  until  March  1986.  Also,  there  was  little  external 
coordination  during  the  unsolicited  proposal  phase 

By  taking  both  extra-organizational  and  intra-orgamzational 
coordination  as  different  subsets  of  information  search  activity,  the 
lack  of  correlation  between  time  and  the  information  search  activity 
can  be  partially  explained.  In  addition,  it  was  originally  expected 
that  the  information  search  subcategory  would  have  behaved  the 
same  as  the  gather  data  subcategory  discussed  above  in  Table  2. 

Both  of  these  similar  subcategories  were  not  found  to  be  related  to 
time.  Therefore,  it  appears  that  in  the  ADF  Competition  case,  the 
Program  Manager  desired  a  comparable  level  of  information  search 
throughout  the  process. 

Returning  to  document  flow,  Table  5  shows  the  results  of  flow 
versus  the  four  acquisition  sub-processes.  The  acquisition 
management  and  contracting  sub-processes  were  both  determined  to 
be  significantly  related  to  the  direction  of  document  flow.  The 
contracting  sub-process  had  a  p-value  of  .0005  while  acquisition 
management  had  a  p-value  of  .0027.  Both  the  requirements  and 
budget  sub-processes  were  very  weakly  related,  with  p-values  of 
.6083  and  .6645  respectively. 

Most  requirements  related  documents  were  either  incoming 
(44.44%)  or  internal  (29.63%).  Since  it  has  already  been  discussed  in 
relation  to  Table  3  that  requirements  documents  seemed  to  cluster 


Contingency  Coefficient/p-value:  Sub-Processes  vs.  Document  Flow 


Requirements 

Budget 

Acq  Mgt 

Contracting 

.102/6083 

.096/6645 

.243/  0027 

.267/  0005 

BOLDFACE:  Statistically  significant  at  .05 

around  the  key  decision  activities  in  April  1985  and  November  1985 
to  January  1986,  this  result  should  be  expected.  Incoming  documents 
were  required  to  deal  with  the  formulation  of  the  unsolicited 
proposal  evaluation  and  for  the  timely  preparation  of  the  draft  RFP 
which  had  to  be  distributed  in  January  1986  to  meet  the 
Congressional  schedule.  Similarly,  intra-orgamzational  coordination 
was  heavily  concentrated  in  November  and  December  1985,  the  same 
time  the  draft  RFP  was  being  prepared. 

That  fact  that  the  requirements  sub-process  was  not  related  to 
flow  was  not  expected.  As  discussed  in  the  first  section  of  this 
chapter,  the  ADF  Program  Manager  was  not  a  key  player  in  the 
requirements  sub-process.  Therefore,  incoming  or  not  applicable 
documents  were  expected  to  dominate  the  activity  in  this  sub¬ 
process.  That  this  was  not  the  case,  indicates  that  program 
managers  can  play  an  important  information  transfer  role  in  the 
requirements  process  and  that,  perhaps  in  this  case  study, 
requirements  related  documentation  was  driven  by  forces  outside  of 
the  requirements  sub-process  itself. 


For  the  budget  sub-process,  most  documents  (35.0%)  were  in 
the  NA  (Not  Applicable)  flow  category.  This  was  probably  due  to  the 
fact  that  most  budget  documentation  was  obtained  by  the  program 
management  team  as  information  copies  of  documents  that  were 
not,  per  se,  meant  for  them  directly,  but  which  contained  funding 
information  necessary  for  effective  acquisition  planning.  The  budget 
documentation  was  expected  to  be  independent  of  the  direction  of 
flow,  and  apparently  was. 

Most  acquisition  management  sub-process  documents  were 
internal  (38.26%)  followed  closely  by  incoming  (37.39%).  Only  12.17%  of 
the  acquisition  management  documents  were  categorized  as  outgoing. 
Since  the  contracting  sub-process  was  also  found  significantly  related 
to  flow,  it  is  necessary  to  address  that  sub-process  along  with  the 
acquisition  management  sub-process.  Internal  documents  were  the 
most  frequent  in  the  contracting  sub-process  (36.43%)  followed  by 
incoming  documents  (27.91%).  The  outgoing  sub-category  was  larger 
for  contracting  at  20.16%.  Given  the  nature  of  this  decision  (i.e. 
considerable  contracting  involvement),  the  amount  of  outgoing 
documentation  appeared  reasonable. 

Much  of  the  coordination  with  outside  organizations  was 
categorized  as  part  of  the  contracting  sub-process.  This  was  also  seen 
in  the  analysis  of  the  observed  frequency  tables  for  those  two  sub¬ 
processes.  The  peak  for  the  acquisition  management  documentation 
was  in  October  1985  before  the  21  November  1985  signing  of  the  ADF 
PMD.  The  peak  for  the  contracting  documentation,  however,  came 


two  months  later  in  December  1985,  just  before  the  release  of  the 
draft  RFP. 

In  summary,  the  Program  Manager  had  a  greater  degree  of 
involvement  in  the  acquisition  management  and  contracting  sub¬ 
processes  than  the  other  two.  That  in  turn  affected  the  timing  and 
high  level  of  internal  coordination  carried  out.  Also,  ADF  program 
management  displayed  the  need  to  coordinate  information  internally 
while  also  receiving  from  other  external  organizations  the 
information  needed  to  accomplish  its  decision  making  task.  This 
finding  substantiates  the  need  for  the  Supporting  Level  of  the  ODAM 
and  supports  the  need  for  the  Organizational  Level  interface. 

Table  6  summarizes  the  comparison  results  of  the  three 
communication  tasks  versus  document  flow.  In  all  three  cases  the 
relationship  was  shown  to  be  significant.  While  the  contingency 
coefficient  of  intra-organization  coordination  was  the  largest,  all 
three  had  p-values  of  .0001.  Therefore,  the  distribution  of 
communication  activity  depended  upon  the  directional  flow  of 
documents  relative  to  the  ADF  program  management  team. 

The  results  reflected  in  Table  6  were  expected.  Internal  and 
outgoing  documents  made  up  48.78%  and  36.59%  respectively  (85.37% 
total)  of  the  information  search  documentation.  Search  initiating 
activity  more  likely  focused  on  functional  areas  within  the 
organization  and  other  interested  external  organizations,  lntra- 
organizational  coordination  accounted  for  73.17%  of  the  internal 
documentation.  Thus,  the  primary  purpose  of  distributing 
information  within  the  organization  was  to  gain  consensus. 


Table  6 


Contingency  Coefficient/p-value:  Communication  Tasks  vs. 

Document  Flow 


Information 

Intra-Org. 

Extra-Org. 

Search 

Coordination 

Coordination 

.365/. OOOl 

.574/0001 

.518/0001 

BOLDFACE:  Statistically  significant  at  .05 

Extra-organizational  coordination  was  dominated  by  incoming 
documents  (52.05%)  and  outgoing  documents  (38.36%).  The  fact  that 
extra-organizational  coordination  documents  were  mostly  incoming, 
indicated  that  many  external  organizations  were  seeking  coordination 
from  the  ADF  Program  Manager.  As  discussed  below,  however, 
there  was  no  relationship  between  any  of  the  sub-processes  and 
extra-organizational  coordination.  Therefore,  while  the  statistical 
significance  of  incoming  extra-organizational  coordination  was  not 
expected,  there  is  not  an  indication  that  any  one  sub-process  drove 
this  result. 

To  complete  the  current  discussion  of  document  flow,  Table  7 
shows  the  results  of  the  contingency  table  analysis  of  flow  versus 
the  five  problem  solving  steps.  Gathering  data  was  the  most 
strongly  related  to  the  flow  (p-value:  .0008)  while  both  the 
identification  of  alternatives  and  evaluation  steps  were  also 
significantly  related  with  a  p-value  of  .0022.  Neither  the  problem 
definition  nor  decision  steps  were  shown  to  be  related  to  flow. 


Table  7 


Contingency  Coefficient/p-value:  Document  Flow  vs. 
Problem  Solving  Steps 


Step 

Problem  Definition 

.100/. 6249 

ID  Alternatives 

.246/  0022 

Gather  Data 

.261/. 0008 

Evaluation 

.246/0022 

Decision 

.169/. 1061 

BOLDFACE:  Statistically  significant  at  .05 

Data  gathering  activity  was  the  most  frequent  of  the  problem 
solving  step  overall  {44.4%)  and  the  most  prevalent  in.  each  of  the 
four  document  flow  categories.  The  largest  single  value  for  any  cell 
of  the  document  flow  versus  problem  solving  step  matrix  was  for 
incoming  documents  supporting  data  gathering  activity  (45.22%).  This 
shows  that  a  primary  objective  of  the  program  management  team 
was  to  gather  information  from  outside  organizations.  Internal 
documents  accounted  for  29.96%  of  the  data  gathering  documents. 

The  decision  step  was  the  least  frequent  problem  solving  step 
with  only  3.86%.  Most  of  the  decision  documents  were  either 
incoming  (40.0%)  or  outgoing  (40.0%).  As  a  percentage  of  all 
documents,  the  outgoing  flow  category  was  more  frequent.  This  is 
reasonable  given  the  involvement  of  the  management  team  in  the 
contracting  sub-process  and  the  need  to  inform  other  organizations  of 
the  evolving  nature  of  the  RFP. 
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Alternative  identification  document  frequency  (16.6%)  was 
mostly  the  result  of  internal  (37.21%)  communication.  Problem 
definition  (15.83%)  was  limited  primarily  to  incoming  (34.15%)  and 
internal  (39.02%)  communications.  Evaluative  documents  were  found 
mostly  in  internal  documents  (37.21%)  although  both  incoming  (27.91%) 
and  outgoing  (27.91%)  also  played  a  large  role.  These  results  imply 
that,  as  expected,  much  of  the  detailed  alternative  identification  and 
evaluation  work  in  the  ADF  Competition  was  managed  relatively 
closely  by  the  ADF  Program  Manager 

The  results  in  Table  7  again  point  to  the  need  for  the  ADF 
Program  Manager  to  interface  with  the  external  environment.  In 
this  case,  external  organizations  were  a  frequent  source  of 
information.  Similarly,  the  complex  ADF  Competition  required 
frequent  internal  communications,  particularly  in  support  of 
alternative  identification,  evaluation,  and  data  gathering  information. 

Table  8  looks  at  the  problem  solving  steps  again  but  shows  the 
results  of  a  comparison  with  the  three  communication  tasks. 

As  Table  8  shows,  the  only  significant  correlation  between  the 
communication  tasks  and  the  problem  solving  steps  was  with 
information  search  activity.  The  search  activity  was  correlated  with 
the  problem  solving  steps  of  identifying  alternatives,  gathering  data 
and  evaluating  the  alternatives.  Gathering  data  was  the  most 
strongly  correlated  with  a  p-value  of  .0001.  Both  the  identifying 
alternatives  and  evaluating  steps  had  a  p-value  of  .0004.  This  last 
result,  and  the  previous  significant  correlations  between  document 
flow  and  information  search  (Table  6)  and  between  document  flow 


Contingency  Coefficient/p-value:  Communication  Tasks  vs 

Problem  Solving  Steps 


Information 

Intra-Org. 

Extra-Org. 

Search 

Coordination 

Coordination 

Problem  Definition 

.012/. 9732 

.030/. 7645 

.043/6103 

ID  Alternatives 

.228/. 0004 

.069/. 3486 

088/  2138 

Gather  Data 

.243/. 0001 

.023/  8199 

.089/1907 

Evaluation 

.228/. 0004 

069/. 3486 

088/  2138 

Decision 

.085/. 3446 

.079/  3549 

008/  8163 

BOLDFACE:  Statistically  significant  at  .05 

these  same  three  problem  solving  steps  (Table  7)  strongly  supports 
that  the  ADF  program  management  team  needed  to  search  out 
information,  from  both  internal  and  external  organizations,  to  assist 
in  the  efforts  to  define  and  evaluate  alternatives  while  also 
gathering  data  to  support  those  tasks. 

The  results  in  Table  8  also  indicate  that  the  other  two 
communication  tasks,  intra-organizational  coordination  and  extra- 
organizational  coordination,  were  not  found  to  be  related  to  any  of 
the  five  problem  solving  steps.  Therefore,  both  intra-organizational 
and  extra-organizational  coordination  were  equally  important 
regardless  of  the  problem  solving  step.  Coordination  was  then  an 
overall  management  problem  solving  objective.  Finally,  returning  to 
the  results  in  Table  6,  the  coordination  objective  was  primarily 
handled  internally  through  normal  intra-organizational 


communications  and  was  handled  externally  through  a  combination 
of  incoming  and  outgoing  correspondence. 

Table  9  shows  the  results  of  comparing  the  communication 
tasks  with  the  four  acquisition  sub-processes. 


Table  9 

Contingency  Coefficient/p-value:  Sub-Processes  vs. 
Communication  Tasks 


Requirements 

Budget 

Acq  Mgt 

Contracting 

Information  Search 

.077/  3346 

.126/. 0810 

087/  2111 

.235/  0002 

Intra-Organization 

Coordination 

.015/4917 

.027/  8586 

.080/. 2463 

.059/  4083 

Extra-Organization 

Coordination 

.038/6932 

091/2192 

.004/9420 

101/1312 

BOLDFACE:  Statistically  significant  at  .05 


Only  the  contracting  sub-process  is  correlated  significantly  with 
any  of  the  three  communication  tasks.  The  p-value  of  contracting 
compared  with  information  search  is  .0002.  This  particular  result 
was  expected.  The  issuance  of  a  RFP  is  a  complex  undertaking.  The 
PM  must  solicit  from  all  interested  parties  their  needs  with  respect 
to  the  contents  and  requirements  to  be  included  in  the  RFP. 

The  remaining  three  sub-processes  were  not  correlated  with 
any  of  the  communication  tasks.  Since  the  ADF  Program  Manager 
was  not  an  active  player  in  the  requirements  sub-process,  it  was 
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expected  that  there  would  not  be  a  significant  relationship.  Likewise, 
the  budget  sub-process  was  working  without  the  active  participation 
of  the  ADF  PM.  As  stated  above,  budget  documents  were  obtained 
from  other  sources  for  information  purposes  to  support  their 
program  planning  efforts. 

Acquisition  management  documentation  was  expected  to 
correlate  with  the  two  coordination  tasks.  However,  common 
acquisition  management  documents  such  as  the  Program 
Management  Plan,  the  Test  &  Evaluation  Master  Plan,  etc.  received 
almost  no  documented  reference.  Again,  the  fact  that  they  did  not 
was  an  indication  of  the  relatively  high  importance  of  the 
contracting  sub-process,  and  its  documentation  requirements,  in  the 
ADF  Competition. 

It  was  expected  that  the  coordination  tasks  would  have  been 
correlated  with  one  or  more  of  the  acquisition  sub-processes  and  at 
least  one  of  the  problem  solving  steps.  As  discussed  above  in  relation 
to  Tables  4  and  6,  the  coordination  activities  were  correlated  with 
both  document  flow  and  time.  Therefore,  while  they  were  an 
important  activity,  the  coordination  documents  purpose  were  not 
identifiable  with  any  one  sub-process,  although  the  search  for 
information  was  closely  tied  to  the  contracting  sub-process. 

Table  10  shows  the  comparison  results  of  the  acquisition  sub¬ 
processes  versus  the  problem  solving  steps.  The  results  are  not 
unlike  what  was  expected. 

The  requirements  sub-process  was  correlated  with  the  problem 
definition  documentation  (p- value:  .0001).  The  comparison  of 


Contingency  Coefficient/p- value:  Sub-Processes  vs 
Problem  Solving  Steps 


Requirements 

Budget 

Acq  Mgt 

Contracting 

Problem  Definition 

.284/  0001 

.100/. 1858 

.193/  0025 

124/0640 

ID  Alternatives 

.019/  9699 

.021/  9738 

.104/  1275 

.286/0001 

Gather  Data 

.026/  8233 

.020/9262 

.087/. 1980 

.122/0621 

Evaluation 

.019/. 9699 

.021/  9738 

.104/. 1275 

.286/  0001 

Decision 

.067/. 5737 

.015/7203 

063/  4863 

0.00/. 7471 

BOLDFACE:  Statistically  significant  at  05 

requirements  in  the  other  four  steps,  however,  was  very  weak  (p- 
values  of  .5737  to  .9699).  It  was  expected  that  the  requirements 
sub-process  would  support  the  problem  definition  step  tasks.  In  a 
case  such  as  the  ADF  Competition,  where  the  problem  could  be 
defined  as  which  plane  did  the  Air  Force  want  to  purchase, 
information  from  the  requirements  sub-process  which  defined  what 
the  organization  needed,  would  contribute  to  the  definition  of  the 
problem.  The  requirements  sub-process  was  not  expected  to 
correlate  with  the  other  problem  solving  steps. 

The  budget  sub-process  was  not  related  to  any  of  the  problem 
solving  steps.  This  was  consistent  with  what  was  discussed  above 
with  respect  to  document  flow;  the  program  management  team 
tended  to  react  to  the  budget  sub-process  instead  of  interacting  with 
it.  This  was  a  function  of  the  fact  that  funds  were  provided  by  the 
Congress  out  of  existing  F-16  program  funding.  There  appeared  to  be 


little  for  the  ADF  team  to  do  except  plan  within  those  funding  limits. 
In  a  program  involved  in  POM  preparation  and  submission,  the 
budget  sub-process  might  be  expected  to  be  correlated  with 
identifying  alternatives,  gathering  data,  and  perhaps  evaluation. 

Problem  definition  activities  were  the  only  problem  solving  step 
correlated  with  the  acquisition  management  sub-process  (p-value: 
.0025).  The  absence  of  correlation  of  the  acquisition  management 
sub-process  in  the  other  steps  reflects  the  relative  importance  of  the 
contracting  documentation  over  that  of  the  acquisition  management 
process  as  mentioned  above. 

The  contracting  sub-process  was  strongly  correlated  to  two  of 
the  problem  solving  steps  -  alternative  identification  (p-value:  .0001) 
and  evaluation  (p-value:  .0001).  Although  more  likely  to  be  the 
result  of  chance,  it  should  be  noted  that  contracting  was  also  close  in 
both  problem  definition  (p-value:  .064)  and  data  gathering  (p-value: 
.0621).  These  results  indicate  that  during  the  ADF  competition  the 
Air  Force  implicitly  relied  upon  the  contracting  process  in  four  of  the 
five  problem  solving  steps  with  help  during  problem  definition  from 
the  requirements  and  acquisition  management  sub-process. 

In  summary,  Table  5  through  Table  10  discussed  various 
subcategory  comparisons  that  were  intended  to  focus  on  the  actual 
workings  of  the  ADF  Competition  decision  making  process.  First,  the 
flow  of  documents  was  related  to  the  four  acquisition  sub-processes, 
the  communication  tasks  and  the  five  problem  solving  steps.  Next, 
the  communication  tasks  were  compared  with  the  problem  solving 
steps  and  then  the  acquisition  sub-processes.  Finally,  the  problem 
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solving  steps  and  the  acquisition  sub-processes  were  compared.  The 
results  of  these  comparisons  showed  that  the  contracting  sub-process 
dominated  the  ADF  management  documentation.  Contracting 
documents  supported  information  search  activity  and  provided 
identification  of  alternatives  and  alternative  evaluation  information. 
The  contracting  sub-process  was  found  to  be  dominant  with  respect 
to  internal  coordinating  activity  and  incoming  documents  from 
external  organizations.  The  internal  documentation  flow  was  a 
probable  reaction  to  the  technical  complexity  of  the  ADF  program 
while  the  external  coordination  was  probably  due  to  the  complex 
management  structure  and  powerful  political  influences.  Overall, 
coordination  appeared  important  as  the  primary  means  used  to  deal 
effectively  with  the  complex  situation  facing  the  ADF  Program 
Manager. 

Table  11  begins  the  discussion  of  the  ability  of  the  overall 
weapon  system  acquisition  process  to  support  the  decision  analysis 
cycle.  Specifically,  Table  11  compares  the  eight  decision  analysis  tasks 
with  the  subcategory  time. 

Only  two  of  the  decision  analysis  tasks  were  found  to  be 
significantly  related  to  time.  Like  other  subcategories  discussed 
earlier,  these  had  frequency  peaks  in  the  October  through  December 
1985  time  period.  These  results  cannot,  however,  be  relied  upon  for 
additional  inferences.  Of  the  eighteen  total  cells,  the  preferences 
subcategory  showed  six  with  an  expected  frequency  of  less  than  one. 
For  the  subcategory  value,  all  eighteen  cells  had  an  expected  value  of 


Table  11 


Contingency  Coefficient/p-value:  Document  Flow  vs.  Time 


Decision  analysis  task 

Outcomes 

.268/  3195* 

Preferences  (See  text) 

.336/. 0147 

Variables 

.257/. 4238* 

Models 

.295/. 1247* 

Probability 

.272/  2838* 

Uncertainty 

.297/1176* 

Value  (See  text) 

.430/. 0001 

Risk 

.224/7422* 

BOLDFACE:  Statistically  significant  at  .05 

*:  Invalid  test  due  to  expected  frequencies  <  1 

less  than  one.  For  the  other  tasks,  only  a  few  cells  had  an  expected 
frequency  of  more  than  one,  thus  making  the  test  invalid. 

Like  Table  11,  some  subcategories  in  Table  12  had  cells  with 
expected  frequency  of  less  than  one.  For  outcomes,  preferences, 
variables,  model,  and  risk,  the  data  element  for  unknown  flow  (i.e. 
unable  to  identify)  had  to  be  adjusted  out  of  the  calculations.  For 
the  three  remaining  subcategories,  probability,  uncertainty,  and 
value,  the  test  was  not  valid  since  all  cells  had  an  expected 
frequency  of  less  than  one. 

The  outcome  definition  task  was  not  correlated  significantly 
with  document  flow.  Of  all  of  the  decision  analysis  tasks,  outcomes 
was  the  most  frequent  categorized  as  NA,  or  received  indirectly. 
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Table  12 


i 


Contingency  Coefficient/p-value:  Document  Flow  vs. 
Decision  Analysis  Tasks 


Decision  analysis  task 

Outcomes 

.087/7393 

Preferences 

.302/. OOOl 

Variables 

.167/. 1136 

Models 

.217/. 0125 

Probability 

.091/7051* 

Uncertainty 

137/  2891* 

Value 

.149/. 2105* 

Risk 

123/. 4108 

BOLDFACE:  Statistically  significant  at 

.05 

*:  Invalid  test  due  to  expected  frequencies  <  1 

The  statement  of  preference  task  was  significantly  related  to 
flow  with  a  p-value  of  .0001.  Incoming  documents  provided  most  of 
the  preference  information  (49.38%)  followed  by  internal  documents 
(34.57%).  This  coincides  with  findings  above  (Table  5)  that  internal 
and  incoming  documents  were  the  primary  documents  used  in  both 
the  acquisition  management  and  contracting  sub-processes.  Also, 
Table  6  showed  that  internal  documents  were  used  for  information 
search  activity  along  with  outgoing  requests  to  external 
organizations.  The  gathering  of  data  was  also  related  to  information 
search  (Table  8).  Given  a  small  lag  for  preparation  and  transrrrssion 
time,  the  incoming  information  about  preferences  can  be  seen  as  a 
effort  within  the  acquisition  management  and  contracting  sub- 
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processes  to  solicit  this  information  from  external  organizations.  This 
conclusion  holds  as  well  for  the  internal  request  for  preference 
information.  These  results  imply  the  following  with  respect  to 
ODAM:  preference  information  was  available  to  the  managerial 
level,  the  interface  with  the  organizational  level  was  an  important 
consideration,  and  that  the  supporting  level  was  used. 

Most  of  the  references  relating  to  model  information  were 
found  in  incoming  (42.86®)  and  outgoing  (33.33%)  documents.  Above, 
information  search  was  shown  to  be  related  to  the  problem  solving 
task,  of  evaluation  of  alternatives  (Table  8)  and  the  contracting  sub¬ 
process  (Table  9).  Later,  in  Table  10,  evaluation  of  alternatives  was 
found  to  be  related  to  the  contracting  sub-process.  Finally,  similar  to 
the  modeling  information,  many  of  the  contracting  documents  were 
categorized  as  incoming  (27.91%).  Therefore,  during  the  ADF 
Competition  there  was  a  pattern  of  requesting  modeling  information 
from  external  organizations  to  support  evaluations  as  part  of  the 
contracting  sub-process. 

It  should  be  remembered  at  this  point  that  the  primary 
benefits  managers  receive  from  decision  analysis,  in  addition  to  its 
explicitness  and  inherent  structuring  of  decision  making,  is  the  ability 
to  deal  effectively  with  future  uncertainty  and  risk.  The  results 
discussed  above  are  important  with  respect  to  the  possible  ability  of 
the  overall  weapon  system  acquisition  process  to  support  decision 
analysis  as  it  is  incorporated  into  the  ODAM.  Specifically,  although 
several  expected  frequencies  were  too  low  (  less  than  one)  to  permit 
statistical  inference,  this  fact  alone  emphasizes  the  relative  lack  of 


attention  that  this  critical  decision  making  information  received 
during  the  ADF  Competition. 

To  better  understand  the  possibilities  for  implementing  decision 
analysis  in  the  weapon  system  acquisition  process,  the  next  three 
tables  present  the  results  of  comparisons  between  the  decision 
analysis  tasks  and  the  communication  tasks,  the  problem  solving 
steps,  and  the  four  acquisition  sub-processes. 

Table  13  shows  the  results  of  the  comparison  of  the  three 
communication  tasks  with  the  decision  analysis  tasks.  Only 

Table  13 


Contingency  Coefficient/p-value:  Communication  Tasks  vs. 
Decision  Analysis  Tasks 


Information 

Intra-Org. 

Extra-Org 

Search 

Coordination 

Coordination 

Outcomes 

.099/. 2041 

.025/8947 

.041/6972 

Preferences 

.094/. 1786 

.094/  1668 

.052/. 4887 

Variables 

.255/. 0001 

.106/  1290 

.077/. 2916 

Model 

.319/. 0001 

.005/8586 

.126/0713 

Probability 

.142/  3435* 

.099/. 6208* 

.039/  6283* 

Uncertainty 

.032/. 8613* 

.008/6648 

.078/. 4809 

Value 

.017/7256* 

.099/. 2649 

.087/. 3602 

Risk 

.024/. 9318 

.116/1317 

.072/. 4324 

BOLDFACE:  Statistically  significant  at 

.05 

*:  Invalid  test  due  to  expected  frequencies  <  1 

information  search  documents  provided  support  for  the  decision 
analysis  tasks.  Information  search  was  significantly  correlated  with 


variable  identification  (p-value:  .0001)  and  modeling  (p-vaiue:  .0001) 
These  results  directly  support  the  finding  above  that  information 
search  activity  supported  the  decision  analysis  modeling  task.  In 
addition,  variable  definition  information  was  also  requested  during 
the  information  search  activity.  Note,  again,  that  several 
subcategory  comparisons  were  invalid  due  to  expected  frequencies 
less  than  one. 

Table  14  shows  a  correlation  between  the  problem  solving  steps 
and  several  of  the  decision  analysis  tasks.  Problem  definition  was 
significantly  related  to  the  identification  of  outcomes  (p-value: 

.0099).  Efforts  to  identify  alternatives  were  related  to  the 
subcategories  of  preferences  (p-value:  .0038),  variables  (p-value: 
.0001),  and  models  (p-value:  .0001).  Data  gathering  documents  were 
shown  to  be  significantly  related  to  the  same  three  decision  analysis 
tasks  as  were  evaluation  documents.  Decision  documents  were 
shown  to  be  related  to  probability,  but  the  test  was  invalid  because 
it  contained  expected  frequencies  less  than  one 

Finally,  Table  15  compares  the  eight  identified  decision  analysis 
tasks  with  the  four  acquisition  sub-processes. 

The  requirements  sub-process  was  found  to  support  the 
decision  analysis  tasks  of  determining  preferences  (p-value:  .0004) 
and  defining  outcomes  (p-value:  .0234).  The  budget  sub-process  was 
a  significant  contributor  to  the  outcomes  task  with  a  p-value  of 
.0038.  This  was  not  expected.  The  evidence,  however,  showed  that 
several  documents  that  were  playing  an  important  role  in  the  PPBS 
were  also  important  sources  of  information  for  the  PM  when 


iaDie  in 

Contingency  Coefficient/p-value;  Problem  Solving  Steps  vs. 

Decision  Analysis  Tasks 


Problem 

Definition 

Identify 

Alternatives 

Gather 

Data 

Evaluate 

Decision 

Outcomes 

.178/. 0099 

.051/  6307 

.015/  9892 

.051/. 6307 

.109/  2652* 

Preferences 

.108/. 1138 

.186/. 0038 

.296/. 0001 

.186/. 0038 

006/  7968 

Variables 

.049/  5825 

.270/. 0001 

.210/. 0010 

.270/. 0001 

.020/  8764 

Models 

.052/  5911 

.309/. 0001 

.186/. 0045 

.309/. 0001 

.015/  7203* 

Probability 

.027/  3536* 

.027/. 3636* 

.069/  9081* 

.027/  3636* 

.012/0157* 

Uncertainty 

.054/  8462* 

.029/. 8314* 

.048/  7834 

029/  8314* 

.136/  3610* 

Value 

.091/  3900* 

.014/. 6959* 

.044/  7947 

.014/  6959* 

.028/  4663* 

Risk 

.025/. 9578 

.030/. 9833 

.083/. 3106 

.030/  9833 

.072/  7817* 

BOLDFACE:  Statistically  significant  at  .05 
*:  Invalid  test  due  to  expected  frequencies  <  1 


identifying  possible  outcomes.  A  good  example  of  this  was  the  early 
concern  that  the  F-20  proposal  would  jeopardize  the  F-16  multi-year 
which  was  then  before  the  Congress  for  approval. 

It  was  also  expected  that  the  acquisition  management 
documents  would  be  more  supportive  of  decision  analysis  than  they 
were.  Only  the  modeling  task  would  find  significant  support  from 
the  acquisition  management  sub-process.  The  contracting  sub-process 
was  found  to  be  significantly  related  to  both  the  modeling  and 
variable  identification  task.  Both  of  these  results  are  reasonable 
because  of  the  work  done  in  the  contracting  sub-process  to  define 
evaluation  criteria  and  write  the  Statement  of  Work  (SOW)  for 
inclusion  in  the  RFP 
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Table  15 


Contingency  Coefficient/p-value:  Sub-Processes  vs. 
Decision  Analysis  Tasks 


Requirements 

Budget 

Acq  Mgt 

Contracting 

Outcomes 

.163/  0234 

.203/. 0038 

.015/  9892 

.046/6159 

Preferences 

.226/0004 

.043/. 6490 

.078/. 2605 

.016/8940 

Variables 

,015/  9491 

.033/. 8382 

.101/. 1440 

.166/0110 

Models 

.008/.  8015 

087/  3215 

.148/  0280 

.206/  0014 

Probability 

,021/. 1908* 

.018/. 1213* 

.055/.  9081* 

.062/1.000* 

Uncertainty 

.160/. 0714* 

.078/. 7392* 

.077/. 4596 

.124/1306 

Value 

.135/. 1436* 

.061/  8689* 

.123/1193 

.028/1.000 

Risk 

.005/. 6334* 

.021/7821* 

.043/7249 

.104/. 1748 

BOLDFACE:  Statistically  significant  at  .05 
*:  Invalid  test  due  to  expected  frequencies  <  1 


In  summary,  the  eight  decision  analysis  tasks  were  only 
partially  addressed  in  the  ADF  Competition  documentation.  Four  of 
the  tasks;  probability,  uncertainty,  value  and  risk,  were  not 
significantly  related  to  any  subcategories  at  all. 

Of  the  four  tasks  remaining,  preferences  and  models  were 
related  to  document  flow.  Preferences  were  found  most  in  incoming 
(49. 3831)  then  internal  (34.57%)  documents  while  modeling  information 
was  found  most  in  incoming  (42.86%)  then  outgoing  (33.33%) 
documents.  The  correlation  of  preferences  with  the  incoming  and 
internal  flows  substantiates  the  need  to  interface  with  the  larger 
organization  and  to  involve  multiple  functions  within  the  unit.  Both 
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of  these  activities  sought  to  gain  organizational  consensus  concerning 
preferences. 

Modeling  was  also  correlated  with  information  search. 

Modeling  information  was  requested  in  29.27£  of  the  information 
request  documents.  The  variables  task  was  also  correlated  with 
information  search.  Information  search  documents  requested 
information  about  decision  variables  in  40.02>  of  those  documents. 

Documents  associated  with  the  five  problem  solving  steps  were 
correlated  with  only  four  of  the  decision  analysis  tasks.  Documents 
concerned  with  defining  the  problem  were  correlated  to  the 
identification  of  outcomes.  The  identification  of  alternatives  step 
correlated  with  three  tasks:  preferences,  variables,  and  models. 
Gathering  data  and  evaluation  also  correlated  with  preferences, 
variables  and  models.  The  problem  solving  steps,  therefore,  resulted 
in  information  being  generated  that  would  allow  the  creation  of  only 
a  simple  deterministic  model.  Outcomes  were  defined,  preferences 
were  stated  and  models  and  variables  were  defined  with  which  to 
evaluate  alternative  solutions.  This  approach,  however,  was  founded 
totally  upon  an  assumption  of  a  certain  future.  There  was  no 
information  relating  to  uncertainty  and  risk.  Also,  the  five  problem 
solving  steps  did  not  elicit  from  the  organization  a  statement  of  its 
relative  values  for  the  various  outcomes. 

The  requirements  sub-process  provided  information  on 
outcomes  and  preferences.  The  budget  sub-process  also  provided 
outcomes  information.  The  acquisition  management  sub-process 
provided  modeling  information  while  contracting  provided 


information  on  both  variables  and  modeling.  Therefore,  the  four 
sub-processes  provided  information  to  address  only  basis  development 
(Step  1)  and  deterministic  structuring  (Step  2)  of  the  decision  analysis 
cycle 

In  summary,  four  decision  analysis  tasks  were  found  correlated 
with  the  sub-processes.  However,  it  appeared  that  the  ADF 
Competition  decision  making  process  virtually  ignored  future 
uncertainty,  value  and  risk  in  its  activities.  Also,  when  searching 
for  additional  information,  only  data  on  variables  or  modeling  were 
sought.  Overall,  while  several  aspects  of  the  decision  analysis  cycle 
were  addressed  in  the  ADF  Competition  decision  process,  the  ability  of 
the  sub-processes,  the  five-step  problem  solving  approach,  and 
normal  coordination  cycles  to  provide  information  useful  for 
incorporating  into  stochastic  models  and  analyzing  organizational 
attitudes  toward  risk  was  missing. 

Analysis  Results  and  Summary 


The  remainder  of  this  chapter  synthesizes  the  results  of  the 
categorization  and  the  statistical  analysis  of  the  program 
documentation.  The  intent  was  to  identify  possible  evidence  of 
applicability  of  the  ODAM  to  the  decision  making  tasks  of  the 
acquisition  Program  Manager  and  also  to  determine  if  ODAM  should 
be  modified  to  account  for  actual  circumstances.  The  synthesis 
follows  directly  the  ODAM  structure  as  presented  in  Chapter  IV. 
First,  the  organizational  level  characteristics  will  be  discussed 


followed  by  the  managerial  level.  Lastly,  the  supporting  level 
elements  of  the  normative  decision  making  model  will  be  discussed. 
Organizational  Level.  The  Organizational  Level  is  summarized 

in  Figure  18.  First  the  inputs  will  be  discussed,  then  the  process,  and 
finally  the  outputs. 


Figure  18 

Organization  Level  Summary 


The  structure  of  the  ADF  Competition  decision  making  unit  was 
hierarchical.  For  the  purposes  of  this  study,  the  focus  was  on  the 
acquisition  PM  located  in  ASD/TA-2.  This  was  a  position  within  the 
organization  set  up  especially  for  the  purpose  of  managing  the  ADF 
Competition.  Depending  on  the  acquisition  sub-process  being 
discussed,  the  relative  position  of  ASD/TA-2  within  the  hierarchy  in 
terms  of  decision  making  authority  varied  somewhat. 

In  a  briefing  given  by  AFSC/CC  to  the  DEPSECDEF  on  20 
January  1986,  the  structure  for  the  contracting  sub-process  proposal 
evaluation  was  defined  as  shown  in  Figure  19. 


Figure  19 


ADF  Source  Selection  Organization 


The  Source  Selection  Authority  (SSA)  was  the  Secretary  of  the 
Air  Force  (SAF).  This  was  not  unusual  for  a  program  of  this 
magnitude  and  political  sensitivity.  For  example,  the  SAF  was  also 
the  SSA  for  the  Advanced  Tactical  Fighter  (ATF)  Demonstration  and 
Validation  program  source  selection  which  was  also  being  completed 
at  the  same  time  as  the  ADF  Competition. 

The  Source  Selection  Advisory  Council  (SSAC)  was  a  body  with 
the  Commander  of  ASD  as  Chairman  which  was  to  provide  the  final 
recommendation  to  the  SSA.  The  SSAC  was  made  up  of 
representatives  from  various  major  commands  as  shown  in  the 
figure. 

The  detail  work  to  be  accomplished  by  the  Source  Selection 
Organization  was  the  responsibility  of  the  Source  Selection  Evaluation 
Board  (SSEB).  The  SSEB  was  chaired  by  ASD/TA.  The  five  specialized 
functional  panels  that  made  up  the  SSEB  were  the  Mission 
Effectiveness  Panel,  the  Technical  Panel,  the  Life  Cycle  Cost  Panel,  the 
Logistics/Management  Panel  and  the  Contracts  Negotiation  Panel. 

The  first  four  of  the  panels  corresponded  to  the  four  evaluation  areas 
defined  in  the  Information  to  Offerors  (ITO).  The  Contracts 
Negotiation  Panel  was  responsible  for  the  actual  conduct  of  the 
negotiation  phase  of  the  source  selection  up  to  the  final  award  of  the 
contract  to  the  winner  of  the  competition. 

ASD/TA-2  was  the  Director,  Air  Defense  Aircraft  Procurement 
Program.  Assigned  to  ASD/TA,  this  individual  was  ASD/TA’s  primary 
assistant  for  all  matters  dealing  with  the  ADF  Competition.  It  was 
ASD/TA-2  who  signed  most  routine  documentation  associated  with 


the  program  and  who  handled  the  bulk  of  the  ADF  management 
team  administrative  tasks.  ASD/TA  maintained  a  similar  position 
during  the  Dual  Role  Fighter  (DRF)  Competition.  At  that  time  it  was 
called  ‘TA-r.  Both  positions  provided  a  visible  focal  point  within  the 
organization  for  all  matters  pertaining  to  these  special  activities. 

Ultimately,  the  definition  of  Program  Manager  used  in  this 
study  must  be  split  between  ASD/TA  and  ASD/TA-2.  While  ASD/TA- 
2  appeared  responsible  for  day-to-day  management  of  the 
competition  program,  virtually  all  documentation  pertaining  to 
direction  of  the  program,  particularly  outgoing  documents,  were 
signed  by  ASD/TA.  Therefore,  when  discussing  the  Managerial  Level 
in  the  next  subsection,  it  will  be  from  the  acquisition  management 
perspective  of  ASD/TA,  with  the  acknowledgement  that  many  of  the 
specific  tasks  might  have  been  delegated  to  an  individual  like 
ASD/TA-2. 

The  structure  for  the  requirements  sub-process  centered  on 
AF/RDQ  (Director  for  Operational  Requirements).  Other  key  players 
in  this  sub-process  were  AFSC/SD  and  T AC/DR  (Deputy  Chief  of  Staff 
for  Requirements).  Also  involved  were  ASD/TA,  NGB/CF  (National 
Guard  Bureau),  and  ADCOM/XP  (Air  Defense  Command).  ASD/TA 
played  an  important  role  even  though  not  a  key  participant.  In  the 
drafting  and  review  of  the  SON,  both  TAC  and  the  Air  Staff  were 
concerned  that  the  SON  reflect  reasonable  requirements  in  terms  of 
the  two  primary  contenders  while  not,  a  priori,  eliminating  any 
others  (e.g.  F-4,  F-18,  etc.). 


In  a  similar  vein,  ASD/TA  was  not  critical  in  the  budget  sub¬ 
process.  The  ADF  Competition  was  funded  from  the  F-16  program. 
The  amounts  per  year  and  the  years  included  were  a  given.  This 
funding  information  was  determined  by  the  Congress.  When  the 
PMD  was  signed  on  21  November  1985,  the  funding  line  was  given  as 
shown  in  Table  16. 


Table  16 

ADF  Program  Funding  Profile 


Appropriation 

FY87 

FY88 

FY89 

FY90 

FY91 

To 

Complete 

3010 

429.3 

664.0 

733.5 

742.1 

749.3 

TBD 

3600 

28.0 

0 

0 

0 

0 

0 

Quantity 

($  in  millions) 

30 

60 

60 

60 

60 

30 

The  acquisition  management  sub-process  centered  on  both 
ASD/TA  and  Director  of  Operational  Requirements  in  the  Air  Staff 
(AF/RDQ)  with  a  significant  role  played  by  AFSC/SD  (Deputy  Chief  of 
Staff  for  Systems).  Most  documented  activity  during  the  period 
April  1985  until  December  1985  in  the  acquisition  management  sub¬ 
process  dealt  with  the  attempts  of  the  Air  Force  to  issue  program 
management  direction  for  the  ADF  Competition.  As  discussed  in 


Chapter  II,  the  critical  document  in  this  early  stage  of  any  program 
is  the  Program  Management  Directive  (PMD).  The  PMD  for  the  ADF 
Competition  was  issued  by  AF/RDQ  on  21  November  1985. 

Coordination  activity  concerning  the  PMD  primarily  centered  around 
the  desire  on  the  Air  Force's  part  to  meet  stated  Congressional 
requirements  for  a  full  and  open  competition  to  be  completed  by  1 
July  1986  and  the  often  conflicting  requirements  of  existing  law  and 
regulations  which  govern  how  the  Air  Force  must  manage  major 
weapon  system  procurements.  The  role  ASD/TA  played  in  this 
process  appeared  to  be  one  of  the  single  conduit  through  which 
information  flowed  between  the  ASD  functional  staff,  the  contractors, 
the  using  command  (TAC),  and  the  Air  Staff.  AF/RDQ,  not  ASD/TA, 
however,  was  the  driving  force  in  this  sub-process. 

In  the  ADF  Competition,  ASD/TA  played  a  central  role  in  only 
the  contracting  sub-process.  This  was  expected  since  contracting  was 
the  most  prevalent  activity  found  in  the  documentation.  It  is 
apparent,  however,  that  there  was  not  a  single  simple  organizational 
structure  at  work  within  the  Air  Force  at  this  time  to  deal  with  the 
ADF  issue.  Four  sub-processes  were  at  work  simultaneously  and 
each  sub-process  had  its  own  focal  point.  Therefore,  a  task  critical  to 
the  success  of  the  program  was  the  ability  of  ASD/TA  to  differentiate 
among  the  sub-processes  while  ensuring  that  all  tasks,  regardless  of 
the  sub-process,  contributed  effectively  to  the  accomplishment  of  the 
overall  organizational  task,  which  was  the  successful  completion  of  a 
’full  and  open'  ADF  competition. 
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The  reliance  upon  established,  formal  procedures  was  evident 
in  all  phases  of  the  competition.  Within  each  sub-process,  the 
participants  were  aware  of  and  often  made  reference  to  the 
regulatory  requirements  concerned  with  the  task  they  were 
performing. 

This  was  particularly  true  of  the  contracting  sub-process.  The 
Federal  Acquisition  Regulation  (FAR)  strictly  governed  the  conduct  of 
procurements  for  major  weapon  systems  by  the  Federal 
government.  In  his  20  January  1986  briefing  to  the  DEPSECDEF, 
AFSC/CC  devoted  an  entire  slide  to  a  comparison  of  the  stages  of  the 
procurement,  the  major  activities  within  each  stage  and  the 
governing  statutes  and  regulations.  Referenced  were  several 
statutes,  the  FAR,  two  AF  regulations,  the  FY86  Continuing 
Resolution  Authority  (CRA)  law  passed  by  the  Congress,  and  waivers 
and  deviations  required  in  order  to  meet  other  requirements  (e  g. 
schedule).  As  mentioned  above,  waivers  to  procedural  schedule 
requirements  were  necessary  because  the  FY86  CRA  required 
"...  the  Air  Force  to  make  source  selection  by  1  July  1986  with 
contract  award  to  follow  in  sixty  days."  Normal  operations  m 
accordance  with  the  established  laws  and  regulations  would  not  have 
permitted  meeting  that  date, 

Little  is  mentioned  in  the  documentation  of  the  personnel  used 
in  the  decision  making  process.  Only  one  individual  is  brought  in  and 
mention  made  of  his  qualifications.  ASD/TA  recommended  to  ASD/CC 
on  26  September  1985  that  he  would  personally  head  the  ADF 
Competition  program  but  would  use  ASD/TA-2  as  'project  officer 
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ASD/TA-2  was  chosen  based  on  operational  flying  experience  in  air 
defense,  prior  acquisition  experience,  and  related  job  experience  at 
the  Air  Staff. 

Environmental  influence  came  from  primarily  the  Congress, 
the  DOD  and  the  Office  of  the  Secretary  of  the  Air  Force  (OSAF). 
Congressional  influence  came  in  the  form  of  the  FY86  Authorization 
Act  and  the  FY86  Continuing  Resolution  Authority. 

The  FY  86  Authorization  Act  stated  that  " . . .  during  FY86  a 
competition  for  the  procurement  of  fighter  aircraft . . .  among  all 
suitable  aircraft... "  should  be  established.  Additionally,  the 
competition  was  to  be  accomplished  in  accordance  with  all  applicable 
provisions  of  law  including  reference  to  sections  from  Title  10,  USC, 
relating  to  the  Director  of  IOT&E  (Section  136a),  to  independent  cost 
estimation  (Section  139c),  and  to  competition  in  contracting  (Section 
137).  These  requirements,  coupled  with  the  FY86  CRA  requirement 
for  a  1  July  1986  source  selection  decision  mentioned  above,  provided 
the  single  most  important  element  the  the  early  planning  had  to 
overcome 

Environmental  influence  from  within  the  DOD  and  OSAF  came 
in  the  form  of  oversight  and  review  by  staff  elements  In  addition 
to  the  normal  reviews  associated  with  these  sub-processes  (e.g. 
Defense  Resources  Board  and  the  Defense  System  Acquisition  Review 
Council),  other  groups  were  established  to  guide  the  ADF  Competition. 
For  example,  a  1  November  1985  memorandum  from  the  Assistant 
Secretary  of  the  Air  Force  for  Research,  Development  and  Logistics 
fSAF/AL)  established  an  ad  hoc  group  to  "  examine  the 


factors/elements  which  will  form  the  basis  of  the  cost  determination 
in  [the]  competition"  (Cooper,  1985).  The  purpose  for  such  a  group 
was  to  ensure  that  cost  information  made  available  to  the  SSA  was 
appropriate  and  comparable. 

Some  aspects  of  the  organizational  level  inputs  made  the 
decision  making  process  easier:  well  structured  organizational 
relationships  and  a  clearly  defined  hierarchy  of  decision  making 
authority,  extensive  procedural  guidelines  to  ensure  the  many 
aspects  of  the  decision  were  adequately  addressed,  and  personnel 
available  to  handle  the  tremendous  workload  involved  with  a  major 
system  source  selection.  However,  some  aspects  of  the  inputs  made 
the  process  less  easy  to  manage:  there  were  no  clearly  defined 
structural  boundaries,  the  procedures  within  each  sub-process  did  not 
always  track  with  progress  in  the  other  sub-processes,  and  the 
environment  provided  constraints  upon  the  decision  making  process 
that  were  not  consistent  with  the  many  requirements  existing  in 
the  sub-processes. 

The  process  at  the  organizational  level  in  the  ADF  Competition 
clearly  reflected  the  coordinate,  compromise,  and  consensus  model  m 
the  ODAM.  In  the  contracting  and  acquisition  management  sub¬ 
processes,  the  25  September  1985  message  from  HQ  AFSC/PK,  Subject 
Air  Defense  Fighter  Competition,  addressed  participants  at  the  Air 
Staff,  ASD,  HQ  TAC,  HQ  AFLC,  the  Logistics  Operations  Center,  and  the 
AF  Acquisition  Logistics  Center.  The  purpose  of  this  early 
communication  was  to  begin  a  ”...  round  table  discussion  on  the 
business  strategies  for  the  Air  Defense  Fighter  Competition."  Such  a 


document  is  evidence  or  the  overall  organizational  process  ot  seeking 
a  consensus  from  all  relevant  participants  as  to  the  best  direction  m 
which  to  proceed 

Similarly,  at  the  direction  of  ASD/CC,  ASD/TA  set  up  a  team  to 
manage  the  ADF  Competition  As  part  of  his  efforts  to  form  this 
team,  ASD/TA  solicited  from  the  larger  organization  outside  ASC 
inputs  from  other  interested  offices  The  nature  of  these  inputs  was 
to  serve  as  "devii's  advocates  ASD/TA  stated  in  a  series  T  3 
October  1985  letters  to  these  devil  $  advocates  that  his  goal  ,s  to 
resoive  as  many  of  these  issues  m  advance  as  we  can  sc  we  :ar. 
clearly  articulatle;  (and!  resoivfe!  critical  programmatic 
issues  '  Furthermore,  the  ASD/TA  stated  objective  was  to 

reduce  the  total  number  of  unresolved  issues  prior  to  briefing 
the  senior  leadership  of  the  Air  Force'  Teal,  1985) 

Other  coordinating  activity  occurred  during  the  program  ou*  is 
categorized  as  supporting  ievei  coordination  This  activity  will  be 
discussed  below 

Outputs  at  the  organizational  level  were  primarily  the  product 
of  the  individual  sub-processes  The  major  product  ot  the 
requirements  sub-process  was  TAF  SON  310-85,  Air  Defense  Fighter 
The  Program  Management  Directive  issued  on  21  November  1985 
served  as  the  primary  output  of  the  acquisition  management  sub¬ 
process 

The  budget  sub-process,  while  less  visible  in  this  uecision 
making  cycle,  resulted  m  changes  to  the  Five  Year  Defense  Program 
(FYDP)  and  on  7  January  1986,  a  Program  Budget  Decision  (PBD) 
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signed  by  the  DEPSECDEF,  detailing  the  funding  requirements  of  the 
ADF  Competition  The  budget  sub-process,  as  part  of  the  PPBS, 
produces  documents  which  most  systematically  relate  to  plans 
(program  descriptions  and  costs),  action  (documents  that  actually 
provide  funding  to  implementing  organizations),  and  control  (tracking 
of  obligation  and  expenditure  rates) 

The  contracting  sub-process  produced  the  RFP  and  later  the 
contract  By  their  very  nature,  these  two  documents  provided  for  a 
plan  of  action  and  the  means  to  control  that  action  In  addition,  the 
contracting  sub-process  produced  several  other  subsidiary  documents 
^hat  also  served  the  purposes  of  planning,  acting,  and  controlling 
For  example,  a  Source  Selection  Plan  (SSP)  was  written  and 
coordinated,  as  was  the  Instructions  to  Offerors  (ITO)  which 
accompanied  the  RFP 

In  summary,  the  Organizational  Level  of  the  ODAM  accurately 
represents  the  decision  making  system  at  work  throughout  the  ADF 
Competition  The  inputs  of  structure,  procedures,  and  the 
environment  all  were  significant  in  their  effect  upon  the  conduct  of 
the  decision  process  Only  the  personnel  element  was  not  frequently 
mentioned  in  the  documentation,  although  the  one  time  it  was 
mentioned,  it  involved  a  pivotal  position  in  the  decision  making  unit 

Drucker's  coordinate,  compromise,  and  consensus  paradigm 
clearly  was  important  A  wide  cross-section  of  Air  Force 
organizations  were  participants  in  the  decision  making  process. 
Finally,  the  bureaucratic  nature  of  the  Air  Force  resulted  in  several 
documents  that  fulfilled  the  output  requirements  of  the  ODAM. 
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Plans  were  written  and  approved,  action  was  taken  contingent  upon 
authorizing  and  justifying  documentation,  and  control  was  exercised 
via  program  management  reviews  and  the  normal  functioning  of  the 
PPBS. 

Managerial  Level.  The  Managerial  Level  activities  are 
summarized  below  in  Figure  20.  The  Managerial  Level  was  much 
less  well  represented  in  the  ADF  Competition  documentation  than 
was  the  Organizational  Level.  Insofar  as  the  Managerial  Level 
focused  upon  the  Program  Manager,  this  implies  that  either  a 
structured  and  explicit  decision  making  framework  as  reflected  in  the 
ODAM  was  lacking,  or  that  another  such  framework  was  used  in  its 
place. 


Figure  20 

Managerial  Level  Summary 
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As  mentioned  above,  the  9  September  1985  message  from 
ASD/CC  to  AFSC/CC  addressed  this  issue  directly.  The  message  stated 
that  there  "...  [did  not]  seem  to  be  any  formal,  structured  dialogue 
on  the  subject"  (McMullen,  1985)  of  what  to  do  about  the  F-20.  The 
message  went  on  to  state  that  much  of  the  information  circulating 
did  not  fully  consider  the  ramifications  and  veracity  of  program 
proposals.  ASD/CC  stated  that  without  a  formal  and  structured 
process  for  making  basic  programmatic  decisions,  there  was  a  chance 
that  the  "wrong  decisions"  might  be  made. 

The  decision  analysis  cycle  introduced  in  Chapter  II  makes  up 
the  Managerial  Level.  Only  aspects  of  Step  1  of  the  cycle,  the 
development  of  the  decision  basis,  were  addressed  in  the  ADF 
Competition. 

Step  1  of  the  decision  analysis  cycle,  Basis  Development,  benefits 
from  the  documents  reflecting  organizational  preferences  for  different 
aspects  of  the  ADF  Competition.  The  fact  that  preferences  were  so 
often  stated  is  a  function  of  the  fact  that  a  primary  purpose  of 
intra-organizational  and  extra-organizational  communications  was  to 
transmit  an  organization's  preference  for  a  given  topic  under 
consideration.  This  activity  then  supports  the  decision  analysis  cycle. 

Deterministic  Structuring,  Step  2  of  the  decision  analysis  cycle, 
was  supported  in  the  ADF  Competition  by  the  documents  dealing 
with  the  identification  of  decision  relevant  variables  and  modeling. 

The  third  most  frequently  documented  decision  analysis  task 
was  the  identification  of  outcomes.  This  activity  supported  basis 
Development  in  tfv5  decision  analysis  cycle.  Next  most  frequent  was 
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documentation  referring  to  models.  Since  very  little  documentation 
reflected  any  concern  for  the  uncertainty  inherent  in  the  decision, 
this  activity  must  be  related  primarily  to  Step  2,  Deterministic 
Structuring.  If  uncertainty  is  taken  into  account  in  the  computer 
modeling  discussed  in  the  documentation,  it  is  not  apparent  in  the 
documentation.  The  Program  Manager  in  such  a  situation  would  not 
be  able  to  articulate  to  others,  much  less  defend,  a  position  based 
upon  an  assumption  of  future  likelihood  of  an  outcome  Information 
concerning  uncertainty  was  not  available  to  the  decision  maker 

Risk  is  another  aspect  of  decision  analysis  which  must  now  be 
addressed.  The  use  of  the  concept  of  risk  m  the  documentation 
varies.  In  a  15  April  1985  memo  to  ASD/CC,  the  Deputy  for  ASD/TA 
states  that  “ .  as  a  result  of  [the  F-20  proposal]  we  may  lose  the  F- 
16  multi-year."  Such  an  outcome  would  adversely  affect  force 
structure  and  would  raise  the  per  unit  cost  of  the  F-16s  bought 
under  the  contract  In  the  2  May  1985  response  to  AFSC/SD 
presenting  the  ASD  evaluation  of  the  Northrup  unsolicited  proposal, 
mention  was  made  of  the  "  .  risk  associated  with  achieving  [the] 
levels  of  R&M"  proposed.  It  is  unclear  whether  this  statement 
referred  to  that  fact  that  it  was  uncertain  if  the  F-20  could  actually 
attain  the  levels  of  R8.M  proposed  or  whether  the  Air  Force  had  to 
acknowledge  the  chance  that  an  outcome  less  than  that  desired  may 
occur. 

Similarly,  a  5  December  1985  memo  from  ASD/TA  to  ASD/CC, 
presents  the  results  of  a  meeting  at  which  ADF  Competition  issues 
were  discussed  among  members  of  the  highest  management  levels 
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[  within  the  Air  Force  (SAF/AL,  USAF/CV,  AF/RD,  TAC/CC,  AFSC/CC, 

I  AFLC/CC  and  ASD/CC)  Issues  were  stated  by  each  of  the  participants 

\ 

!  as  implicit  desires  to  avoid  possible  negative  outcomes  which  might 

result  from  the  competition  For  instance,  SAF/AL  was  concerned 
that  the  acquisition  business  strategy  should  allow  the  F-20  to 
compete  USAF/CV  was  concerned  that  the  Air  Force  comply  with 
Congressional  direction  while  at  the  same  time  maintaining  the 
integrity  of  the  tactical  fighter  budget 

Step  3  of  the  decision  analysis  cycle,  Probabalistic  Evaluation, 
was  therefore  not  well  evidenced  in  the  documentation  Sensitivity 
analysis,  whether  deterministic  or  probabalistic,  was  not  apparent  at 
all 

As  discussed  in  Chapter  11,  Step  A  Basis  Appraisal  deals  with  the 
evaluation  of  the  basis  in  terms  of  the  value  of  information  If  the 
value  of  additional  information  exceeds  the  benefits  to  be  gained 
from  that  information,  then  it  is  time  to  make  the  decision  There 
was  no  evidence  that  any  consideration  was  given  to  the  vaiue  of 
information  The  information  search  activity  undertaken  was 
primarily  driven  by  procedural  requirements  or  in  reaction  to  the 
stated  requirements  and  desires  of  other  organizational  elements  or 
powerful  constituencies  (e  g  Congress) 

The  breakdown  of  information  search  activity  among  the  sub¬ 
processes  was  found  to  be  requirements  at  A  2%,  acquisition 
management  at  29  2%,  contracting  at  66  7%,  and  budget  at  0%  No 
information  search  was  undertaken  m  the  budget  sub-process  and 
very  little  was  done  during  the  requirements  sub-process 
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As  previously  discussed,  the  bulk  of  information  search 
documentation  concerned  preferences  (32.7$),  variables  (26.9$),  and 
modeling  (23  1$)  This  search  activity  coincided  with  either  requests 
from  other  organizations  for  their  vie’  .-$  to  be  considered  or  from 
the  procedural  requirements  for  inputs  from  other  organizations 
(external)  or  functions  (internal)  The  documentation  does  not 
support  a  determination  that  the  ADF  Competition  management,  or 
the  sub-process  procedural  requirements,  explicitly  considered  the 
value  of  information  in  the  decision  making  process 

In  summary,  the  Managerial  Level  did  not  appear  to  reflect  a 
well  structured  decision  making  process  as  portrayed  in  the  ODAM 
Evidence  that  another  such  structured  process  was  used  is  also  not 
evident  The  management  process  at  work  appears  to  be  one  of 
reacting  to  requirements  from  outside  the  decision  making  unit  (e  g 
Congress  primarily) 

Supporting  Level  The  Supporting  Level  is  summarized  in 
Figure  21  Internal  coordination  of  documents  began  early  in  the 
ADF  Competition  While  somewhat  limited  during  the  first  phase  of 
the  competition  (i  e  evaluating  and  responding  to  the  Northrup 
unsolicited  proposal)  to  organizational  elements  immediately  affected 
by  the  issue  and  the  tasking  for  the  evaluation  (e  g.  AF/RDQ, 
ASFC/SD,  and  ASD/CC/TA),  by  early  October  1985,  more  formal 
support  was  required  from  a  broader  range  of  organizational 
interests  and  functions. 

In  a  3  October  1985  letter  to  various  ASD  organizations,  ASD/TA 
requested  support  for  the  formation  of  an  ADF  Competition  team 


Figure  21 

Supporting  Level  Summary 


This  team  formed  the  nucleus  of  ASD  intra-organization  coordination 
throughout  the  remainder  of  the  program.  The  team  consisted  of 
ASD/TA-2,  ASD/T  AAE/T  A  AT /T  A  AK/T  A  AL/T  A  AF,  ASD/TAL,  ASD/TAD, 
ASD/TAK,  ASD/TAP,  ASD/YZF,  ASD/TACSO-A,  AFLC/JANA,  and 
ASD/YPRD 

Also,  in  key  documents  being  submitted  to  ASD/CC  for 
approval,  the  concurrence  of  other  key  ASD  managers 
(ASD/YP/AV/PM/AC/YZ)  was  obtained  Finally,  documents  routed  to 
ASD/CC  either  for  information  or  for  approval  were  routinely 
submitted  through  the  ASD/CC  staff  -  ASD/CS  and  ASD/CV 

In  summary,  the  relevance  of  the  Supporting  Level  of  the 
ODAM  was  substantiated  in  the  ADF  Competition.  Also,  the 
documentation  reflected  that  supporting  level  activities  were  an 
important  part  of  the  actual  management  of  the  ADF  decision 
making  process 
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Summary 

This  chapter  compared  the  Air  Defense  Fighter  Competition 
with  the  Organizational  Decision  Analysis  Model  (ODAM)  developed  in 
Chapter  IV.  First,  an  historical  overview  of  the  competition  was 
presented  beginning  with  the  receipt  of  the  Northrup  unsolicited 
proposal  on  2  April  1985  until  contract  award  was  announced  on  31 
October  1986  Next,  the  results  of  the  categorization  and  statistical 
analysis  of  the  ADF  Program  Manager's  documentation,  according  to 
the  requirements  of  the  ODAM,  were  summarized  and  discussed. 
Finally,  the  ADF  program  documentation  was  analyzed  with  respect 
to  its  correlation  to  the  three  levels  of  the  ODAM 

In  the  next  chapter,  the  entire  study  will  be  summarized, 
conclusions  presented  and  recommendations  made. 
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VI.  Summary  and  Conclusions 

Introduction 

Air  Force  acquisition  Program  Managers  (PM)  are  faced  with 
the  task  of  integrating  a  complex  mix  of  goals,  objectives  and 
procedures  from  four  sub-processes  within  the  major  weapon  system 
acquisition  process.  This  complexity  is  compounded  by  the  dual  roles 
of  the  Program  Manager:  traditional  manager  coupled  with  primary 
program  advocate. 

The  many  rules  and  procedures  of  the  four  sub- processes 
constitute  a  complex  milieu  for  allocating  the  resources  under  the 
control  of  the  acquisition  Program  Manager.  This  complexity  is  the 
result  of  the  necessary  fragmentation  of  tactical  objectives  and 
operational  rules  between  the  sub-processes.  This  fragmentation 
affects  the  ability  of  the  Air  Force  to  efficiently  make  a  decision  and 
then  to  implement  that  decision  as  effective  action.  Managerial 
decision  making  efficiency  requires  a  structured  information 
generation  process  while  implementation  effectiveness  requires 
explicit  and  unambiguous  communication  of  relevant  facts. 

This  study  has  shown  that  the  Air  Force  lacks  an  overall 
conceptual  organizational  decision  making  framework  for  the 
acquisition  Program  Manager.  Such  a  framework  would  efficiently 
and  effectively  guide  the  PM  faced  with  these  fragmented  decision 
making  rules  and  procedures.  While  decision  analysis  appeared  to 
offer  a  means  of  solving  this  problem,  the  technique  had  not  yet 
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been  applied  to  organizational  decision  making  situations  similar  to 
that  of  the  weapon  system  acquisition  Program  Manager 

Conclusions 

The  purpose  of  this  study  was  to  develop  a  normative  decision 
making  model  to  integrate  and  guide  decision  making  processes  in 
large  bureaucratic  organizations  like  the  Air  Force  There  were  two 
main  problems  to  overcome.  First,  was  to  understand,  in  as  broad  a 
perspective  as  possible,  the  complexities  inherent  in  the  Air  Force 
acquisition  decision  making  process  to  include  the  internal  tasks  that 
a  PM  was  required  to  accomplish  and  the  external  forces  under 
whose  influence  these  tasks  were  to  be  done  Second,  was  to  define 
how  to  integrate  a  decision  making  methodology,  intended  for  use 
only  in  individual  decision  making  situations,  into  a  large  complex 
group  decision  making  situation  in  the  Air  Force  bureaucracy 

The  overall  objective  of  this  research  was  to  provide  better 
understanding  of  the  organizational  decision  making  processes  in  the 
Air  Force  acquisition  arena  in  terms  of  information  generation  and 
communication.  Specifically,  the  study  sought  to: 

1.  develop  a  normative  decision  making  model  for 
use  in  the  weapon  system  acquisition  management 
process  which  integrated  decision  analysis  principles 
and  procedures, 

2.  test  this  model  against  an  actual  program 
management  decision,  and 


3.  draw  conclusions  about  the  implementation  of 
the  model  into  acquisition  management  procedures. 

The  primary  work  necessary  to  accomplish  the  study  objective 
was  divided  into  two  parts.  First,  the  results  of  the  literature 
review  were  used  to  create  a  model  of  organizational  decision 
making  incorporating  the  principles  of  decision  analysis  -  the 
Organizational  Decision  Analysis  Model  (ODAM).  Second,  the  Air 
Defense  Fighter  (ADF)  Competition  decision  was  compared  to  the  basic 
attributes  of  the  ODAM.  The  purpose  of  the  comparison  was  to 
identify  whether  the  ODAM  could  improve  the  PM's  decision  making 
in  the  organizational  setting  of  the  DOD  weapon  system  acquisition 
process. 

The  remainder  of  this  section  will  discuss  the  conclusions  and 
recommendations  associated  with  each  of  the  objectives  mentioned 
above. 

Normative  Model  Development.  The  first  objective  of  the 
study  was  to  develop  a  normative  decision  making  model  for  use  in 
the  weapon  system  acquisition  management  process  which 
integrated  decision  analysis  principles  and  procedures.  The 
development  of  the  Organizational  Decision  Analysis  Model  (ODAM) 
met  that  objective. 

As  discussed  in  Chapter  II,  a  previous  limitation  in  the  use  of 
decision  analysis  was  that  it  could  only  be  used  as  a  tool  for 
individual  decision  making.  For  the  acquisition  PM  this  was 
unacceptable.  The  complex  nature  of  weapon  system  program 
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management  dictated  the  involvement  not  only  of  many  people  but 
also  of  many  different  organizations. 

Howard  stated  that  managers  in  organizations  act  on  behalf  of 
the  organization.  This  implies  that  the  ”...  individual  who  as 
principal  devolves  some  decision  making  authority  upon  an  agent  is 
making  a  decision  in  that  devolution . . ,  and  thus,  whether  as 
principal  or  agent,  everyone  is  making  individual  decision  making" 
(Howard,  1983a:  181).  Howard  goes  on  to  say,  however,  that  group 
decision  making  can  be  considered  for  decision  analysis  only  if  the 
individuals  comprising  the  group  are  "acting  collectively  in  accordance 
with  an  agreement"  (Howard,  1983a:  181). 

The  essential  central  point  of  the  ODAM  is  the  provision  for  the 
reaching'  of  a  consensus  (collective  agreement)  both  at  the 
organizational  level  where  strategic  goals  and  objectives  are 
determined  and  at  the  unit,  or  operational,  level  where  the  specific 
expertise  exists  to  generate  information  to  solve  the  problem.  The 
inclusion  in  ODAM  of  Drucker's  coordinate,  compromise,  consensus 
paradigm  answers  Howard's  need  for  collective  agreement.  It  does 
this  by  ensuring  that  the  end  product  of  the  decision  (the  action) 
meets  the  needs  of  the  organization  as  a  whole  and  that  the  detailed 
information  used  during  the  decision  analysis  cycle  is  complete  and 
correct. 

Adequate  coordination  within  the  unit  and  throughout  the 
organization  was  not  sufficient  taken  alone'.  DOD  acquisition  decision 
making  is  not  carried  out  in  isolation.  Therefore,  ODAM  also 
acknowledged  that  the  Program  Manager  must  assess  the  resources 
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the  decision  process  will  utilize  and  the  constraints  it  will  encounter. 
Listed  as  inputs  at  the  organizational  level  of  ODAM,  these  include 
|  the  existing  organizational  structure,  established  procedures,  available 

personnel,  and  the  state  of  the  environment  with  regard  to  the 
specific  decision  to  be  made.  The  nature  of  the  inputs  must  be 
understood  at  the  beginning  of  the  decision  process  and  must  be 

. 

|  reviewed  often. 

The  Program  Manager,  as  choreographer  of  the  decision 
analysis  cycle,  transforms  information  into  an  implemental  decision. 
For  a  decision  to  be  effective,  there  must  be  a  commitment  to  action. 
The  outputs  of  ODAM  address  this  aspect  of  decision  making.  At  an 
j  organizational  level  there  must  be  a  plan  that  says  what  will  be 

done,  which  then  generates  action  as  a  result  of  the  decision,  and 
which  can  be  effectively  controlled  by  management  using  the  explicit 
and  unambiguous  information  provided  by  the  decision  analysis 
cycle. 

However,  the  weapon  system  acquisition  process  is  dynamic  as 
well  as  complex.  This  dynamic  or  changing  nature  implies  that  the 
future  is  not  certain.  In  addition,  weapon  system  development  and 
acquisition  programs  can  now  last  ten  years  or  more.  It  is 
impossible  to  predict  what  will  occur  one  year  in  the  future  much 
less  ten  or  fifteen  years.  Decision  analysis  came  into  being  to  deal 
not  only  with  the  complexity  discussed  above,  but  also  to  allow 
managers  to  deal  with  the  future  and  the  uncertainty  inherent  in 
it.  Uncertainty  in  decision  analysis  is  represented  by  subjective 
probabilities.  These  probabilities  are  based  upon  the  judgments  of 
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experts  and/or  accepted  by  the  decision  makers.  Attitudes  toward 
risk  are  assessed  which  capture  the  feelings  of  the  organization 
toward  the  possible  negative  outcomes  that  might  result  from  the 
decision. 

Explicit  values  for  each  outcome  are  also  agreed  upon  so  that 
unambiguous  quantitative  comparisons  of  alternatives  can  be 
accomplished  and  reviewed.  The  result  of  this  process  is  that  more 
informed  and  defensible  decisions  are  made  thus  improving  the 
effectiveness  of  the  Program  Manager  and  the  organization  as  a 
whole. 

Since  it  has  been  shown  that  the  ODAM  can  provide  structure 
to  problem  solving,  improve  information  generation  and 
communication  activities,  and  deal  effectively  with  complexity  and 
uncertainty,  it  is  recommended  that  the  Air  Force  implement 
training  programs  addressing  normative  models  of  organizational 
decision  making.  Specifically,  the  ODAM  should  be  included  in  the 
Professional  Military  Education  (PME)  curricula  to  expose  managers 
to  management  techniques  of  decision  making  processes  that  offer  a 
broader  view  of  the  complex  organizational  processes  at  work  during 
the  decision  making  process.  This  would  help  managers  efficiently 
create  explicit  documentation  of  all  information  relevant  to  the 
decision  to  be  made,  while  accepting  future  uncertainty  as  a  reality 
and  providing  a  means  to  effectively  deal  with  it. 

A  second  recommendation  is  that  the  Air  Force  Institute  of 
Technology  prepare  and  teach  graduate  level  and  Professional 
Continuing  Education  (PCE)  courses  that  educate  Air  Force  managers 
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in  organizational  decision  making.  At  the  graduate  level,  these 
courses  should  be  inter-disciplinary  and  bring  together  relevant  topics 
to  include  organizational  structure,  organizational  effectiveness, 
decision  analysis,  subjective  probability  theory,  decision  support 
system,  and  the  use  of  automated  tools  (e.g.  simulations)  to  support 
organizational  decision  making.  Part  of  this  program  should  be 
frequent  interchange  of  ideas  between  the  management  practitioners 
in  the  School  of  Systems  and  Logistics  and  the  technical  specialists  in 
the  School  of  Engineering.  For  example,  once  a  quarter,  an  informal 
student/faculty  symposium  could  be  held  at  which  time  students  and 
faculty  would  have  the  opportunity  to  present  organizational  decision 
making  related  research  proposals,  findings  and  recommendations  for 
discussion  by  the  group. 

In  PCE  courses,  AFIT  should  present  to  students  (especially 
program  managers  in  SYS  100,  SYS  200,  and  SYS  400)  the  basic 
objectives  of  the  0DAM  and  how  they  could  implement  it  in  their 
own  offices  to  make  their  decision  making  more  efficient  and 
effective.  For  the  less  experienced  program  managers,  the 
complexity  issue  and  how  0DAM  deals  with  it  should  be  introduced. 
For  more  experienced  managers,  the  issues  of  organizational  values, 
uncertainty  and  risks  could  be  introduced.  In  both  cases,  however, 
the  documentation  and  communication  tasks  implied  by  the  model 
must  be  stressed. 

A  third  recommendation  is  that  AFIT  thesis  research  be 
managed  within  the  broad  area  of  organizational  decision  making  to 
produce  a  richer  understanding  of  both  the  problems  to  be  overcome 


in  implementing  a  framework  such  as  QDAM  and  the  possibilities 
available  to  Air  Force  managers  to  improve  the  effectiveness  of  their 
decision  making  role. 

Comparative  Analysis  of  Model.  The  study’s  second  objective 
was  to  test  the  ODAM  against  an  actual  program  management 
decision.  This  objective  was  met  by  comparing  the  program 
management  documentation  from  the  Air  Force's  recent  Air  Defense 
Fighter  (ADF)  Competition  decision  to  the  ODAM.  There  were  two 
sub-objectives  applicable  to  the  case  study.  First,  the  case  study  was 
to  provide  a  solid  and  realistic  foundation  as  the  practical  point  of 
reference  for  drawing  conclusions  about  the  applicability  of  the 
ODAM.  Secondly,  it  was  desirable  to  determine  if  existing  procedures 
within  the  acquisition  system  provided  the  kinds  of  information 
required  by  decision  analysis  techniques. 

The  first  sub-objective  was  accomplished  by  categorizing  the 
ADF  documentation  in  accordance  with  the  methodology  defined  in 
Chapter  III  and  summarized  in  Figures  10  and  11.  Next,  the  data 
from  the  categorization  were  statistically  analyzed  for  correlation 
between  relevant  sets  of  subcategories.  The  most  prominent  results 
of  this  analysis  are  discussed  next. 

The  statistical  analysis  first  investigated  the  subcategory  time. 
The  intent  was  to  understand  the  relationship  of  the  documents  to 
the  key  points  during  the  decision  process:  the  unsolicited  proposal 
review  (2  April  -  2  May  1985),  the  issuance  of  the  ADF  Program 
Management  Directive  (PMD)  on  21  November  1985,  the  validation  of 
the  ADF  Statement  of  Need  (SON)  on  6  January  1986,  and  the  release 


on  24  March  1986  of  the  ADF  Request  for  Proposal  (RFP).  Documents 
from  three  of  the  sub-process  were  correlated  to  time:  budget, 
acquisition  management  and  contracting.  This  was  expected  since 
the  unique  activities  required  by  each  sub-process  would  necessarily 
precede  the  major  documents  they  produced. 

The  requirements  sub-process  was  an  exception  to  this.  It  was 
expected  that  activity  to  define  requirements  would  occur  early  in 
the  decision  cycle.  This  expectation  was  reinforced  by  results 
showing  that  documents  categorized  as  problem  definition, 
preferences,  and  value  were  found  to  be  correlated  with  time  and 
exhibited  an  observed  frequency  early  in  the  cycle.  The  fact  that 
requirements  differed  indicated  that  the  Program  Manager  used 
supplemental  information  to  support  the  early  decision  process. 
Therefore,  the  requirements  sub-process  was  not  fully  able  to  fulfill 
its  function,  although  the  nature  of  the  decision  initiation  (i.e. 
unsolicited  proposal)  provided  a  ready  explanation  why  this  was  the 
case. 

Each  of  the  four  sub-processes,  in  general,  displayed  correlation 
with  a  different  mix  of  subcategories.  The  requirements  sub-process 
did  provide  information  that  helped  define  the  problem,  identify 
outcomes  and  state  preferences  for  outcomes.  Although  problem 
definition  and  preferences  were  correlated  with  time  as  well, 
outcomes,  like  the  requirements  sub-process,  was  not.  Therefore,  it 
appeared  that  the  ADF  Competition  PM  did  not  seek  to  independently 
identify  possible  outcomes  of  the  decision. 
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The  budget  sub-process  was  not  active  in  the  decision  process, 
at  least  not  from  the  perspective  of  the  ADF  program  manager. 
Although  budget  documents  were  correlated  with  both  time  and  the 
identification  of  outcomes,  it  was  noted  that  budget  documents  were 
mostly  in  the  flow  subcategory  element  ‘not  applicable’  (i.e. 
indicating  an  indirect  and  undefined  routing  to  the  PM).  Therefore, 
it  v/as  likely  that  the  correlation  to  time  was  driven  by  the  PPBS 
cycle  and  not  the  ADF  Competition  decision  process. 

The  acquisition  management  sub-process  was  correlated  to  time 
and  to  document  flow.  Like  the  budget  sub-process,  it  followed  the 
pattern  of  a  substantial  peak  in  the  number  of  documents 
transmitted  in  the  critical  October  through  December  1985  period. 
Internal  acquisition  management  documents  were  most  prevalent 
and  incoming  documents  second. 

The  contracting  sub-process,  however,  was  the  greatest 
contributor  to  the  ADF  Competition  decision  making  process.  It  was 
as  part  of  the  contracting  sub-process  that  the  Program  Manager 
handled  36.43%  of  the  internal  documentation,  27.91%  of  the  incoming 
documentation,  and  sent  out  20.16%  of  all  documents.  Contracting 
was  the  only  sub-process  to  correlate  with  the  information  search 
task  as  well  as  identification  of  alternatives,  and  alternative 
evaluation  problem  solving  steps.  The  contracting  sub-process  also 
provided  information  about  modeling  and  made  brief  mention  of 
probability. 

Therefore,  the  Program  Manager  relied  primarily  upon  the 
contracting  sub-processes  with  lesser  support  from  the  acquisition 
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management  sub-process.  The  nature  of  the  procedural 
requirements  of  these  two  sub-processes  determined  the  timing  and 
high  level  of  internal  coordination  carried  out  during  the  decision 
making  process. 

The  ADF  PM  also  demonstrated  the  need  to  coordinate 
information  internally  while  also  receiving  from  other  external 
organizations  the  information  needed  to  accomplish  its  decision 
making  task.  This  finding  substantiates  the  need  for  the  Supporting 
Level  of  the  ODAM  and  supports  the  need  for  the  Organizational 
Level  interface. 

The  level  of  document  flow  was  found  to  be  significantly 
related  to  time.  Internal  documents,  for  instance,  would  tend  to  be 
more  frequent  in  some  months  rather  than  others.  Similarly,  intra- 
organizational  coordination  was  found  to  be  related  to  time.  These 
results  underscore  the  importance  of  gaining  an  internal 
organizational  consensus.  The  observed  frequency  tables  show  this  to 
be  particularly  so  in  the  unsolicited  proposal  evaluation  phase  and  in 
the  three  month  peak  period  just  before  the  draft  RFP  release. 

Extra-organizational  coordination  was  also  correlated  to  time. 
However,  it  exhibited  a  peak  in  the  level  of  documentation  that 
differed  from  intra-organizational  coordination.  Extra-organizational 
coordination  was  lacking  in  the  April  -  May  1985  period  but 
continued  after  the  October  to  December  1985  peak,  with  higher 
levels  lasting  until  Marcn  1986. 

In  summary,  various  subcategory  comparisons  were 
accomplished  to  focus  on  the  actual  workings  of  the  ADF  Competition 
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decision  making  process.  First,  the  flow  of  documents  was  related  to 
the  four  acquisition  sub-processes,  the  communication  tasks  and  the 
five  problem  solving  steps.  The  contracting  sub-process  dominated 
the  ADF  Competition  program  management  documentation. 
Contracting  documents  supported  information  search  activity  and 
provided  identification  of  alternatives  and  alternative  evaluation 
information.  The  contracting  sub-process  was  found  to  be  most 
active  with  respect  to  internal  coordinating  activity  and  incoming 
documents  from  external  organizations.  The  internal  documentation 
flow  was  a  probable  reaction  to  the  technical  complexity  of  the  ADF 
program  while  the  external  coordination  was  probably  due  to  the 
complex  management  structure  and  powerful  political  influences. 
Overall,  coordination  appeared  important  as  the  primary  means  used 
to  deal  effectively  with  the  complex  situation  facing  the  ADF 
Program  Manager. 

Given  the  strong  compatibility  of  portions  of  the  ADF 
Competition  with  the  structure  of  the  ODAM,  it  is  recommended  that 
the  Air  Force  implement  a  formal  method  to  track  acquisition 
program  documentation  across  sub-processes.  Such  a  tracking 
method,  based  on  the  ODAM,  would  help  the  overall  organization 
deal  with  the  complexity  of  modern  weapon  system  acquisitions  as 
well  as  the  dynamics  of  defense  management  in  general. 

The  objective  of  the  tracking  system  would  be  to  document  the 
agreement  between  the  Program  Manager  and  the  overall 
organization  about  what  the  program  will  be  in  the  future  and 
why.  The  program  management  tracking  system  would  provide  the 
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organisation  with  a  single  document  that  focuses  not  on  what  the 
program  is  (i.e.  a  baseline  or  PMP),  but  instead  defines  for  program 
management  what  the  program  is  intended  to  be.  Anticipated 
benefits  arising  from  the  program  should  be  tracked  back  to  an 
unambiguous  method  (e.g.  a  model)  that  relates  specific  assumptions 
and  agreements  about  goals,  objectives,  alternatives,  values,  methods 
of  evaluation,  and  the  state  of  influential  environmental  factors  to 
desired  outcomes  and  the  tasks  necessary  to  produce  them.  As  in 
the  ODAM,  the  institutionalization  of  such  a  system  would  result  in 
more  relevant  advanced  planning,  more  effective  action,  and  more 
efficient  management  control. 

It  is  also  recommended  that  additional  studies  be  conducted  to 
validate  the  model.  These  studies  should  focus  on  different  types  of 
decisions,  more  precise  variable  definitions ,  and  a  greater  variety  of 
data  sources. 

Implementation  of  Model.  The  third  objective  of  this  study 
was  to  draw  conclusions  about  the  possible  implementation  of  the 
ODAM  into  the  acquisition  management  arena  for  use  by  Air  Force 
program  managers.  This  objective  was  met  by  comparing  the 
category  of  critical  decision  analysis  tasks  with  the  other  categories 
in  the  content  analysis  scheme  (See  Figure  11). 

The  intent  of  this  objective  was  to  determine  whether  there 
was  any  existing  support  (i.e.  information  generation  activities  and 
products)  for  decision  analysis  already  available  in  the  overall 
weapon  system  acquisition  process.  The  results  showed  that  half  of 
the  eight  critical  decision  analysis  tasks  could  be  supported  through 
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existing  procedures,  although  the  ability  to  determine  the  degree  of 
fit  was  not  possible.  Those  tasks  addressed  were:  identification  of 
outcomes,  statements  of  preferences,  definition  of  variables  and  use 
of  quantitative  models.  These  four  tasks  provided  some  of  the 
information  needed  to  support  step  1  (Basis  Development)  and  step  2 
(Deterministic  Structuring)  of  ODAM’s  Managerial  Level  decision 
analysis  cycle. 

Table  17  summarizes  the  comparisons  of  the  four  acquisition 
sub-processes  with  the  eight  decision  analysis  tasks. 

Table  17 

Sub-Process  versus  Decision  Analysis  Tasks 


Requirements 

Budget  Acq.  Mgt  Contracting 

Outcomes  • 

• 

Preferences  • 

Variables 

• 

Models 

•  • 

Probability 

Uncertainty 

Value 

Risk 

•  Correlation  existed 

The  remaining  four  decision  analysis  tasks  were  not  shown  to 
be  correlated  to  any  of  the  sub-processes.  Also,  the  volume  of 
documents  addressing  these  last  four  was  very  small,  sometimes 
only  one  or  two  total  documents.  Also,  it  was  these  tasks  that 
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would  have  provided  the  information  necessary  for  the  PM  to 
understand  the  impact  of  future  uncertainty  on  the  program. 

This  study  did  not  investigate  certain  areas  that  may  have 
contained  information  about  these  tasks.  For  instance,  classified 
documents  were  excluded  from  consideration  and  might  have 
included  statements  about  uncertainty  or  organizational  values. 
Similarly,  computer  simulations  used  to  support  the  ADF  Competition 
could  not  be  included  and  might  have  included  information  about 
probabilities,  values,  etc.  However,  the  fact  remains  that  the  ADF 
documentation  made  virtually  no  reference  to  these  subjects. 
Therefore,  since  the  four  decision  analysis  tasks  of  uncertainty, 
probability,  values,  and  risk  were  not  correlated,  it  appears  that  the 
ADF  Competition  decision  was  made  while  minimizing  the  affects  on 
the  outcomes  of  an  uncertain  future. 

It  is  recommended  that  the  Air  Force  investigate  ways  to 
inject  the  explicit  determination,  specification  and  consideration  of 
uncertainty  and  risk  into  all  management  processes.  Additionally, 
the  information  about  uncertainty  and  risk  that  is  produced  should 
be  made  an  integral  part  of  all  program  documentation  including 
program  review  presentations  to  facilitate  discussions  among  the 
larger  Air  Force  acquisition  community. 

Decision  analysis  inherently  provides  structure  to  data.  This 
structure  increases  understanding  of  the  problem  and  clarifies  what 
decision  is  to  be  made,  thus  focusing  the  Program  Manager's  decision 
making  efforts  and  clarifying  issues  to  the  broader  organization.  In  a 
manner  similar  to  the  Requirements  Correlation  Matrix  in  the 
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recently  revised  SON  procedures,  the  structure  provided  by  the 
ODAM  would  allow  the  PM  to  track  over  time,  in  an  integrated 
fashion,  the  status  of  all  key  decision  variables.  If  a  key  decision 
variable  were  to  change,  it  would  then  be  easier  to  generate  new 
alternatives  and  evaluate  them  consistently  with  earlier  evaluations 
which  would  then  in  turn  increase  the  Program  Manager's  ability  to 
explain  or  justify  any  resulting  technical  or  programmatic  changes  to 
the  program's  constituencies.  This  focus  should  improve  the  quality 
of  both  the  communication  which  accompanies  a  decision  and  the 
specific  action  which  results  from  the  decision. 

Summary 

The  use  of  decision  analysis  techniques  within  an  overall  • 
normative  decision  making  model  can  assist  Air  Force  Program 
Managers  in  allocating  the  scarce  resources  of  money,  time,  and 
manpower  effectively  and  efficiently,  A  framework  such  as  the 
ODAM  provides  the  Program  Manager  with  a  method  to  manage 
complex  processes,  factor  uncertainty  into  decision  making,  make 
communications  more  effective  and  improve  post-decision 
management  control. 

The  ODAM  ensures  that  decisions  are  adequately  defined, 
rationally  consistent,  and  defensible  to  others.  To  the  extent  that 
these  principles  are  implemented  along  side  the  existing  management 
processes  faced  by  the  Program  Manager,  the  overall  management  of 
the  weapon  system  acquisition  process  can  be  improved. 
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Block  19 

■  Abstract 

v'  '  Program  managers  are  faced  with  the  task  of  integrating 
a  complex  mix  of  goals*  objectives  and  procedures  from  four 
sub-processes  within  the  major  weapon  system  acquisition 
process.  These  sub-processes  include  needs  determination, 
budget*  acquisition  management*  and  contracts.  This  complex¬ 
ity  is  compounded  by  the  dual  roles  of  the  program  manager t 
traditional  manager  coupled  with  primary  program  advocate. 

To  be  effective  in  this  organizational  situation,  the 
program  manager  requires  a  framework  to  efficiently  guide 
decision  making  efforts.  This  study  developed  a  normative 
decision  making  model  to  guide  program  managers  in  dealing 
with  this  complexity.  The  model*  based  upon  management 
theory,  focused  on  information  generation  and  communication 
tasks.  It  was  then  compared  to  the  Air  Defense  Fighter 
Competition  decision. 

Decision  analysis  techniques  were  integrated  into  the 
model  to  structure  the  decision  making  process  to  efficiently 
generate  relevant  information  in  a  form  to  maximize  its 
utility  to  the  organization  as  a  whole. 

The  case  study  evaluated  Air  Defense  Fighter  program 
documentation  to  ascertain  applicability  of  the  model. 

Analysis  showed  that  a  valid  normative  model  can  be  developed 
for  use  by  a  program  manager  working  in  a  bureaucratic  r* 
organization!  that  the  various  sub-processes  generated 
some  of  the  information  required  by  decision  analysis 
techniques,  but  that  information  regarding  organizational 
values  and  assessments  of  future  uncertainty  and  risk 
were  not  used  or  requested!  that  coordinating  activities 
were  a  critical  part  of  efforts  to  reach  consensus  concern¬ 
ing  goals*  objectives*  etc.!  and  that  the  sub-processes  did 
not  provide  an  overall  coherent  decision  making  structure.  <  7;. 

Recommendations  were  that  the  Air  Force  initiate 
training  programs  in  organizational  decision  making!  that 
program  managers  be  required  to  structure  decision-making 
activities  to  generate  and  document  information  in 
accordance  with  the  normative  model t  and  that  mithods  to 
explicitly  consider  value,  uncertainty,  and  risk  be 
studied  and  implemented. 
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